Compare commits

..

No commits in common. "a24d49042ba6d52241383589dc7c12788f548cc2" and "3727d60786600cb40634f9fceddcf45fa57a7fa1" have entirely different histories.

24 changed files with 469 additions and 1071 deletions

2
Cargo.lock generated
View file

@ -126,7 +126,7 @@ checksum = "9555578bc9e57714c812a1f84e4fc5b4d21fcb063490c624de019f7464c91268"
[[package]]
name = "ckan-devstaller"
version = "0.3.0"
version = "0.1.0"
dependencies = [
"anyhow",
"clap",

View file

@ -1,6 +1,6 @@
[package]
name = "ckan-devstaller"
version = "0.3.0"
version = "0.1.0"
edition = "2024"
[dependencies]

View file

@ -1,36 +1,42 @@
# ckan-devstaller
<a href="https://ckan-devstaller.dathere.com"><img width="1165" height="668" alt="{1329F0BA-A29F-4BF8-BB6B-E3BA84FDAFCC}" src="https://github.com/user-attachments/assets/8f0cc4ef-d90d-4715-ba21-9083dff0c3ff" /></a>
`ckan-devstaller` attempts to install CKAN 2.11.3 from source using [ckan-compose](https://github.com/tino097/ckan-compose), intended for development use in a new Ubuntu 22.04 instance. The following are also installed and enabled by default:
- [DataStore extension](https://docs.ckan.org/en/2.11/maintaining/datastore.html)
- [ckanext-scheming extension](https://github.com/ckan/ckanext-scheming)
- [DataPusher+ extension](https://github.com/dathere/datapusher-plus)
`ckan-devstaller` attempts to install a [CKAN](https://ckan.org) instance using [ckan-compose](https://github.com/tino097/ckan-compose) for development usage in a new Ubuntu 22.04 instance.
[DRUF mode](https://github.com/dathere/datapusher-plus?tab=readme-ov-file#druf-dataset-resource-upload-first-workflow) is available but disabled by default. The [`datatablesview-plus` extension](https://github.com/dathere/ckanext-datatables-plus) is planned to be included in a future release.
You may find `ckan-devstaller` useful for:
## Quick start
- Exploring CKAN for the first time without spending hours on installation steps
- Developing/Testing CKAN extensions and fixing bugs
- Trying a new CKAN version to test an upgrade from a legacy version
> [!CAUTION]
> Make sure `ckan-devstaller` is run in a **new** Ubuntu 22.04 instance. Do NOT run `ckan-devstaller` in an existing instance that is important for your usage.
`ckan-devstaller` was made to help speed up the installation time for CKAN and various extensions/features to boost development productivity.
> [!WARNING]
> If you are using Ubuntu 22.04 on VirtualBox, you may need to add your user to the sudoers file before running the ckan-devstaller install script. Open a terminal in your virtual machine (VM), run `su -` and log in as the root user with the password you used to set up the VM, then type `sudo adduser <username> sudo` where `<username>` is your username then restart your VM and run the ckan-devstaller installer script.
**Get started at [ckan-devstaller.dathere.com](https://ckan-devstaller.dathere.com).**
> [!NOTE]
> The `/etc/ckan/default/ckan.ini` config file will have its comments removed for now. There are plans to fix this in a future release of `ckan-devstaller`.
## Learn more about developing with CKAN
> [!NOTE]
> Currently `ckan-devstaller` supports x86 architecture. ARM support is planned.
You may find the following guides useful while developing with CKAN:
You have two common options to choose from for installation. Paste one of the following scripts into your new Ubuntu 22.04 instance's terminal.
- [CKAN Hardware Requirements](https://github.com/ckan/ckan/wiki/Hardware-Requirements) - Learn what you need before installing CKAN
- [CKAN Sysadmin guide](https://docs.ckan.org/en/latest/sysadmin-guide.html) - Useful for CKAN instance administrators/sysadmins
- [CKAN Theming guide](https://docs.ckan.org/en/latest/theming/index.html) - Explore how to set up custom themes for your CKAN instance
- [CKAN Extending guide](https://docs.ckan.org/en/latest/extensions/index.html) - Develop CKAN extensions that can enhance your CKAN instance's functionality and add custom features
### Install with non-interactive mode (default config)
## What next?
```bash
wget -O - https://github.com/dathere/ckan-devstaller/releases/download/0.2.1/install.bash | bash -s default
```
- [Customize your config file](https://docs.ckan.org/en/latest/extensions/index.html)
- [Create test data](https://docs.ckan.org/en/latest/maintaining/getting-started.html#creating-test-data)
- [Visit ckan.org](https://ckan.org)
### Install with interactive mode
## `ckan-devstaller` demos
```bash
wget -O - https://github.com/dathere/ckan-devstaller/releases/download/0.2.1/install.bash | bash
```
## Demos
### Interactive customizable installation

View file

@ -38,6 +38,33 @@ export default function HomePage() {
<div className="relative mb-4">
<Hero />
</div>
<Cards>
<Card
icon={<ZapIcon />}
href="/docs/quick-start"
title="Quick start"
>
Get started with ckan-devstaller and install CKAN within minutes
</Card>
<Card icon={<BlocksIcon />} href="/docs/builder" title="Builder">
Customize your installation with an interactive web GUI
</Card>
<Card
icon={<HomeIcon />}
href="/docs/reference/installation-architecture"
title="Installation architecture"
>
Learn about where files are installed after running
ckan-devstaller
</Card>
<Card
icon={<GitMergeIcon />}
href="https://github.com/dathere/ckan-devstaller"
title="Source code"
>
View the source code of ckan-devstaller on GitHub
</Card>
</Cards>
</div>
</main>
</>
@ -45,7 +72,6 @@ export default function HomePage() {
}
function Hero() {
const { Card, Cards } = defaultMdxComponents;
return (
<div className="relative z-2 flex flex-col border-x border-t bg-fd-background/80 px-4 pt-12 max-md:text-center md:px-12 md:pt-16 [.uwu_&]:hidden overflow-hidden">
<div
@ -86,7 +112,7 @@ function Hero() {
</Link>
.
</p>
<div className="inline-flex items-center gap-3 max-md:mx-auto mb-8">
<div className="inline-flex items-center gap-3 max-md:mx-auto">
<Link
href="/docs"
className={cn(
@ -108,33 +134,6 @@ function Hero() {
Source Code
</Link>
</div>
<Cards>
<Card
icon={<ZapIcon />}
href="/docs"
title="Quick start"
>
Get started with ckan-devstaller and install CKAN within minutes
</Card>
<Card icon={<BlocksIcon />} href="/docs/builder" title="Builder">
Customize your installation with an interactive web GUI
</Card>
<Card
icon={<HomeIcon />}
href="/docs/reference/installation-architecture"
title="Installation architecture"
>
Learn about where files are installed after running
ckan-devstaller
</Card>
<Card
icon={<GitMergeIcon />}
href="https://github.com/dathere/ckan-devstaller"
title="Source code"
>
View the source code of ckan-devstaller on GitHub
</Card>
</Cards>
<PreviewImages />
</div>
);
@ -150,7 +149,7 @@ function PreviewImages() {
];
return (
<div className="p-8 min-w-[800px] overflow-hidden xl:-mx-12 dark:[mask-image:linear-gradient(to_top,transparent,white_40px)]">
<div className="mt-12 p-8 min-w-[800px] overflow-hidden xl:-mx-12 dark:[mask-image:linear-gradient(to_top,transparent,white_40px)]">
<div className="absolute flex flex-row left-1/2 -translate-1/2 bottom-4 z-2 p-1 rounded-full bg-fd-card border shadow-xl dark:shadow-fd-background">
{/* <div
role="none"

View file

@ -1,5 +1,4 @@
@import "tailwindcss";
@import "fumadocs-ui/css/neutral.css";
@import "fumadocs-ui/css/ocean.css";
@import "fumadocs-ui/css/preset.css";
@import "tw-animate-css";
@import 'tailwindcss';
@import 'fumadocs-ui/css/neutral.css';
@import 'fumadocs-ui/css/ocean.css';
@import 'fumadocs-ui/css/preset.css';

View file

@ -2,7 +2,6 @@ import "@/app/global.css";
import { RootProvider } from "fumadocs-ui/provider";
import localFont from "next/font/local";
import Script from "next/script";
import { Toaster } from "@/components/ui/sonner";
const inter = localFont({ src: "../lib/inter.ttf" });
@ -19,7 +18,6 @@ export default function Layout({ children }: LayoutProps<"/">) {
data-web-vitals="true"
strategy="afterInteractive"
/>
<Toaster closeButton richColors />
</body>
</html>
);

View file

@ -8,20 +8,17 @@
"@radix-ui/react-collapsible": "^1.1.12",
"@radix-ui/react-tabs": "^1.1.13",
"class-variance-authority": "^0.7.1",
"clsx": "^2.1.1",
"fumadocs-core": "15.8.1",
"fumadocs-mdx": "12.0.1",
"fumadocs-ui": "15.8.1",
"lucide-react": "^0.545.0",
"lucide-react": "^0.544.0",
"next": "15.5.4",
"next-themes": "^0.4.6",
"react": "^19.1.1",
"react-dom": "^19.1.1",
"sonner": "^2.0.7",
"tailwind-merge": "^3.3.1",
},
"devDependencies": {
"@biomejs/biome": "2.2.6",
"@biomejs/biome": "2.2.5",
"@tailwindcss/postcss": "^4.1.13",
"@types/mdx": "^2.0.13",
"@types/node": "24.5.2",
@ -29,7 +26,6 @@
"@types/react-dom": "^19.1.9",
"postcss": "^8.5.6",
"tailwindcss": "^4.1.13",
"tw-animate-css": "^1.4.0",
"typescript": "^5.9.2",
},
},
@ -37,23 +33,23 @@
"packages": {
"@alloc/quick-lru": ["@alloc/quick-lru@5.2.0", "", {}, "sha512-UrcABB+4bUrFABwbluTIBErXwvbsU/V7TZWfmbgJfbkwiBuziS9gxdODUyuiecfdGQ85jglMW6juS3+z5TsKLw=="],
"@biomejs/biome": ["@biomejs/biome@2.2.6", "", { "optionalDependencies": { "@biomejs/cli-darwin-arm64": "2.2.6", "@biomejs/cli-darwin-x64": "2.2.6", "@biomejs/cli-linux-arm64": "2.2.6", "@biomejs/cli-linux-arm64-musl": "2.2.6", "@biomejs/cli-linux-x64": "2.2.6", "@biomejs/cli-linux-x64-musl": "2.2.6", "@biomejs/cli-win32-arm64": "2.2.6", "@biomejs/cli-win32-x64": "2.2.6" }, "bin": { "biome": "bin/biome" } }, "sha512-yKTCNGhek0rL5OEW1jbLeZX8LHaM8yk7+3JRGv08my+gkpmtb5dDE+54r2ZjZx0ediFEn1pYBOJSmOdDP9xtFw=="],
"@biomejs/biome": ["@biomejs/biome@2.2.5", "", { "optionalDependencies": { "@biomejs/cli-darwin-arm64": "2.2.5", "@biomejs/cli-darwin-x64": "2.2.5", "@biomejs/cli-linux-arm64": "2.2.5", "@biomejs/cli-linux-arm64-musl": "2.2.5", "@biomejs/cli-linux-x64": "2.2.5", "@biomejs/cli-linux-x64-musl": "2.2.5", "@biomejs/cli-win32-arm64": "2.2.5", "@biomejs/cli-win32-x64": "2.2.5" }, "bin": { "biome": "bin/biome" } }, "sha512-zcIi+163Rc3HtyHbEO7CjeHq8DjQRs40HsGbW6vx2WI0tg8mYQOPouhvHSyEnCBAorfYNnKdR64/IxO7xQ5faw=="],
"@biomejs/cli-darwin-arm64": ["@biomejs/cli-darwin-arm64@2.2.6", "", { "os": "darwin", "cpu": "arm64" }, "sha512-UZPmn3M45CjTYulgcrFJFZv7YmK3pTxTJDrFYlNElT2FNnkkX4fsxjExTSMeWKQYoZjvekpH5cvrYZZlWu3yfA=="],
"@biomejs/cli-darwin-arm64": ["@biomejs/cli-darwin-arm64@2.2.5", "", { "os": "darwin", "cpu": "arm64" }, "sha512-MYT+nZ38wEIWVcL5xLyOhYQQ7nlWD0b/4mgATW2c8dvq7R4OQjt/XGXFkXrmtWmQofaIM14L7V8qIz/M+bx5QQ=="],
"@biomejs/cli-darwin-x64": ["@biomejs/cli-darwin-x64@2.2.6", "", { "os": "darwin", "cpu": "x64" }, "sha512-HOUIquhHVgh/jvxyClpwlpl/oeMqntlteL89YqjuFDiZ091P0vhHccwz+8muu3nTyHWM5FQslt+4Jdcd67+xWQ=="],
"@biomejs/cli-darwin-x64": ["@biomejs/cli-darwin-x64@2.2.5", "", { "os": "darwin", "cpu": "x64" }, "sha512-FLIEl73fv0R7dI10EnEiZLw+IMz3mWLnF95ASDI0kbx6DDLJjWxE5JxxBfmG+udz1hIDd3fr5wsuP7nwuTRdAg=="],
"@biomejs/cli-linux-arm64": ["@biomejs/cli-linux-arm64@2.2.6", "", { "os": "linux", "cpu": "arm64" }, "sha512-BpGtuMJGN+o8pQjvYsUKZ+4JEErxdSmcRD/JG3mXoWc6zrcA7OkuyGFN1mDggO0Q1n7qXxo/PcupHk8gzijt5g=="],
"@biomejs/cli-linux-arm64": ["@biomejs/cli-linux-arm64@2.2.5", "", { "os": "linux", "cpu": "arm64" }, "sha512-5DjiiDfHqGgR2MS9D+AZ8kOfrzTGqLKywn8hoXpXXlJXIECGQ32t+gt/uiS2XyGBM2XQhR6ztUvbjZWeccFMoQ=="],
"@biomejs/cli-linux-arm64-musl": ["@biomejs/cli-linux-arm64-musl@2.2.6", "", { "os": "linux", "cpu": "arm64" }, "sha512-TjCenQq3N6g1C+5UT3jE1bIiJb5MWQvulpUngTIpFsL4StVAUXucWD0SL9MCW89Tm6awWfeXBbZBAhJwjyFbRQ=="],
"@biomejs/cli-linux-arm64-musl": ["@biomejs/cli-linux-arm64-musl@2.2.5", "", { "os": "linux", "cpu": "arm64" }, "sha512-5Ov2wgAFwqDvQiESnu7b9ufD1faRa+40uwrohgBopeY84El2TnBDoMNXx6iuQdreoFGjwW8vH6k68G21EpNERw=="],
"@biomejs/cli-linux-x64": ["@biomejs/cli-linux-x64@2.2.6", "", { "os": "linux", "cpu": "x64" }, "sha512-1HaM/dpI/1Z68zp8ZdT6EiBq+/O/z97a2AiHMl+VAdv5/ELckFt9EvRb8hDHpk8hUMoz03gXkC7VPXOVtU7faA=="],
"@biomejs/cli-linux-x64": ["@biomejs/cli-linux-x64@2.2.5", "", { "os": "linux", "cpu": "x64" }, "sha512-fq9meKm1AEXeAWan3uCg6XSP5ObA6F/Ovm89TwaMiy1DNIwdgxPkNwxlXJX8iM6oRbFysYeGnT0OG8diCWb9ew=="],
"@biomejs/cli-linux-x64-musl": ["@biomejs/cli-linux-x64-musl@2.2.6", "", { "os": "linux", "cpu": "x64" }, "sha512-1ZcBux8zVM3JhWN2ZCPaYf0+ogxXG316uaoXJdgoPZcdK/rmRcRY7PqHdAos2ExzvjIdvhQp72UcveI98hgOog=="],
"@biomejs/cli-linux-x64-musl": ["@biomejs/cli-linux-x64-musl@2.2.5", "", { "os": "linux", "cpu": "x64" }, "sha512-AVqLCDb/6K7aPNIcxHaTQj01sl1m989CJIQFQEaiQkGr2EQwyOpaATJ473h+nXDUuAcREhccfRpe/tu+0wu0eQ=="],
"@biomejs/cli-win32-arm64": ["@biomejs/cli-win32-arm64@2.2.6", "", { "os": "win32", "cpu": "arm64" }, "sha512-h3A88G8PGM1ryTeZyLlSdfC/gz3e95EJw9BZmA6Po412DRqwqPBa2Y9U+4ZSGUAXCsnSQE00jLV8Pyrh0d+jQw=="],
"@biomejs/cli-win32-arm64": ["@biomejs/cli-win32-arm64@2.2.5", "", { "os": "win32", "cpu": "arm64" }, "sha512-xaOIad4wBambwJa6mdp1FigYSIF9i7PCqRbvBqtIi9y29QtPVQ13sDGtUnsRoe6SjL10auMzQ6YAe+B3RpZXVg=="],
"@biomejs/cli-win32-x64": ["@biomejs/cli-win32-x64@2.2.6", "", { "os": "win32", "cpu": "x64" }, "sha512-yx0CqeOhPjYQ5ZXgPfu8QYkgBhVJyvWe36as7jRuPrKPO5ylVDfwVtPQ+K/mooNTADW0IhxOZm3aPu16dP8yNQ=="],
"@biomejs/cli-win32-x64": ["@biomejs/cli-win32-x64@2.2.5", "", { "os": "win32", "cpu": "x64" }, "sha512-F/jhuXCssPFAuciMhHKk00xnCAxJRS/pUzVfXYmOMUp//XW7mO6QeCjsjvnm8L4AO/dG2VOB0O+fJPiJ2uXtIw=="],
"@emnapi/runtime": ["@emnapi/runtime@1.5.0", "", { "dependencies": { "tslib": "^2.4.0" } }, "sha512-97/BJ3iXHww3djw6hYIfErCZFee7qCtrneuLa20UXFCOTCfBM2cvQHjWJ2EG0s0MtdNwInarqCTz35i4wWXHsQ=="],
@ -499,7 +495,7 @@
"lru-cache": ["lru-cache@11.2.2", "", {}, "sha512-F9ODfyqML2coTIsQpSkRHnLSZMtkU8Q+mSfcaIyKwy58u+8k5nvAYeiNhsyMARvzNcXJ9QfWVrcPsC9e9rAxtg=="],
"lucide-react": ["lucide-react@0.545.0", "", { "peerDependencies": { "react": "^16.5.1 || ^17.0.0 || ^18.0.0 || ^19.0.0" } }, "sha512-7r1/yUuflQDSt4f1bpn5ZAocyIxcTyVyBBChSVtBKn5M+392cPmI5YJMWOJKk/HUWGm5wg83chlAZtCcGbEZtw=="],
"lucide-react": ["lucide-react@0.544.0", "", { "peerDependencies": { "react": "^16.5.1 || ^17.0.0 || ^18.0.0 || ^19.0.0" } }, "sha512-t5tS44bqd825zAW45UQxpG2CvcC4urOwn2TrwSH8u+MjeE+1NnWl6QqeQ/6NdjMqdOygyiT9p3Ev0p1NJykxjw=="],
"magic-string": ["magic-string@0.30.19", "", { "dependencies": { "@jridgewell/sourcemap-codec": "^1.5.5" } }, "sha512-2N21sPY9Ws53PZvsEpVtNuSW+ScYbQdp4b9qUaL+9QkHUrGFKo56Lg9Emg5s9V/qrtNBmiR01sYhUOwu3H+VOw=="],
@ -693,8 +689,6 @@
"shiki": ["shiki@3.13.0", "", { "dependencies": { "@shikijs/core": "3.13.0", "@shikijs/engine-javascript": "3.13.0", "@shikijs/engine-oniguruma": "3.13.0", "@shikijs/langs": "3.13.0", "@shikijs/themes": "3.13.0", "@shikijs/types": "3.13.0", "@shikijs/vscode-textmate": "^10.0.2", "@types/hast": "^3.0.4" } }, "sha512-aZW4l8Og16CokuCLf8CF8kq+KK2yOygapU5m3+hoGw0Mdosc6fPitjM+ujYarppj5ZIKGyPDPP1vqmQhr+5/0g=="],
"sonner": ["sonner@2.0.7", "", { "peerDependencies": { "react": "^18.0.0 || ^19.0.0 || ^19.0.0-rc", "react-dom": "^18.0.0 || ^19.0.0 || ^19.0.0-rc" } }, "sha512-W6ZN4p58k8aDKA4XPcx2hpIQXBRAgyiWVkYhT7CvK6D3iAu7xjvVyhQHg2/iaKJZ1XVJ4r7XuwGL+WGEK37i9w=="],
"source-map": ["source-map@0.7.6", "", {}, "sha512-i5uvt8C3ikiWeNZSVZNWcfZPItFQOsYTUAOkcUPGd8DqDy1uOUikjt5dG+uRlwyvR108Fb9DOd4GvXfT0N2/uQ=="],
"source-map-js": ["source-map-js@1.2.1", "", {}, "sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA=="],
@ -727,8 +721,6 @@
"tslib": ["tslib@2.8.1", "", {}, "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w=="],
"tw-animate-css": ["tw-animate-css@1.4.0", "", {}, "sha512-7bziOlRqH0hJx80h/3mbicLW7o8qLsH5+RaLR2t+OHM3D0JlWGODQKQ4cxbK7WlvmUxpcj6Kgu6EKqjrGFe3QQ=="],
"typescript": ["typescript@5.9.2", "", { "bin": { "tsc": "bin/tsc", "tsserver": "bin/tsserver" } }, "sha512-CWBzXQrc/qOkhidw1OzBTQuYRbfyxDXJMVJ1XNwUHGROVmuaeiEm3OslpZ1RV96d7SKKjZKrSJu3+t/xlw3R9A=="],
"undici-types": ["undici-types@7.12.0", "", {}, "sha512-goOacqME2GYyOZZfb5Lgtu+1IDmAlAEu5xnD3+xTzS10hT0vzpf0SPjkXwAw9Jm+4n/mQGDP3LO8CPbYROeBfQ=="],

View file

@ -1,22 +0,0 @@
{
"$schema": "https://ui.shadcn.com/schema.json",
"style": "default",
"rsc": true,
"tsx": true,
"tailwind": {
"config": "",
"css": "app/global.css",
"baseColor": "neutral",
"cssVariables": true,
"prefix": ""
},
"iconLibrary": "lucide",
"aliases": {
"components": "@/components",
"utils": "@/lib/utils",
"ui": "@/components/ui",
"lib": "@/lib",
"hooks": "@/hooks"
},
"registries": {}
}

View file

@ -1,107 +0,0 @@
import defaultMdxComponents from "fumadocs-ui/mdx";
import { SailboatIcon, TerminalSquareIcon } from "lucide-react";
import { Config, selectedCardClasses } from "../builder";
import { toast } from "sonner";
const getExtensionClassName = (config: Config, extensionName: string) => {
return config.extensions.includes(extensionName) ? selectedCardClasses : "";
};
const updateExtensions = (
config: Config,
setConfig: any,
extensions: string[] | string,
mode?: "add" | "remove",
) => {
const extensionsArray = Array.isArray(extensions) ? extensions : [extensions];
if (mode === "add") {
setConfig({
...config,
extensions: [...new Set([...config.extensions, ...extensionsArray])],
});
return;
}
for (const extensionName of extensionsArray) {
if (config.extensions.includes(extensionName))
setConfig({
...config,
extensions: config.extensions.filter(
(extension) => extension !== extensionName,
),
});
else if (!config.extensions.includes(extensionName))
setConfig({
...config,
extensions: [...config.extensions, extensionName],
});
}
};
export default function CKANExtensionsBuilderSection({
config,
setConfig,
}: {
config: Config;
setConfig: any;
}) {
const { Card, Cards } = defaultMdxComponents;
return (
<>
<h3>CKAN extensions</h3>
<Cards>
<Card
className={getExtensionClassName(config, "ckanext-scheming")}
icon={<TerminalSquareIcon />}
title="ckanext-scheming"
onClick={() => {
if (
config.extensions.includes("DataPusher+") &&
config.extensions.includes("ckanext-scheming")
) {
toast.error(
"You cannot remove the ckanext-scheming extension because the DataPusher+ extension depends on it.",
);
return;
}
updateExtensions(config, setConfig, "ckanext-scheming");
}}
></Card>
<Card
className={getExtensionClassName(config, "DataStore")}
icon={<TerminalSquareIcon />}
title="DataStore"
onClick={() => {
if (
config.extensions.includes("DataPusher+") &&
config.extensions.includes("DataStore")
) {
toast.error(
"You cannot remove the DataStore extension because the DataPusher+ extension depends on it.",
);
return;
}
updateExtensions(config, setConfig, "DataStore");
}}
></Card>
<Card
className={getExtensionClassName(config, "DataPusher+")}
icon={<TerminalSquareIcon />}
title="DataPusher+"
onClick={() => {
if (config.extensions.includes("DataPusher+")) {
updateExtensions(config, setConfig, "DataPusher+");
} else {
updateExtensions(
config,
setConfig,
["DataPusher+", "ckanext-scheming", "DataStore"],
"add",
);
}
}}
></Card>
</Cards>
</>
);
}

View file

@ -1,39 +0,0 @@
import defaultMdxComponents from "fumadocs-ui/mdx";
import { SailboatIcon } from "lucide-react";
import { selectedCardClasses } from "../builder";
export default function CKANVersionBuilderSection({ config, setConfig }: any) {
const { Card, Cards } = defaultMdxComponents;
return (
<>
<h3>CKAN version</h3>
<Cards>
<Card
icon={<SailboatIcon />}
title="2.11.3"
className={
config.ckanVersion === "2.11.3"
? selectedCardClasses
: "cursor-pointer"
}
onClick={() => {
setConfig({ ...config, ckanVersion: "2.11.3" });
}}
></Card>
<Card
icon={<SailboatIcon />}
title="2.10.8"
className={
config.ckanVersion === "2.10.8"
? selectedCardClasses
: "cursor-pointer"
}
onClick={() => {
setConfig({ ...config, ckanVersion: "2.10.8" });
}}
></Card>
</Cards>
</>
);
}

View file

@ -1,48 +0,0 @@
import defaultMdxComponents from "fumadocs-ui/mdx";
import { SailboatIcon, TerminalSquareIcon } from "lucide-react";
import { Config, selectedCardClasses } from "../builder";
const getFeatureClassName = (config: Config, featureName: string) => {
return config.features.includes(featureName) ? selectedCardClasses : "";
};
const updateFeatures = (
config: Config,
setConfig: any,
featureName: string,
) => {
if (config.features.includes(featureName))
setConfig({
...config,
features: config.features.filter((feature) => feature !== featureName),
});
else setConfig({ ...config, features: [...config.features, featureName] });
};
export default function FeaturesBuilderSection({
config,
setConfig,
}: {
config: Config;
setConfig: any;
}) {
const { Card, Cards } = defaultMdxComponents;
return (
<>
<h3>Features</h3>
<Cards>
<Card
className={getFeatureClassName(config, "enable-ssh")}
icon={<TerminalSquareIcon />}
title="Enable SSH"
onClick={() => {
updateFeatures(config, setConfig, "enable-ssh");
}}
>
Installs the openssh-server package.
</Card>
</Cards>
</>
);
}

View file

@ -1,63 +0,0 @@
import { Config, selectedCardClasses } from "../builder";
import { BarChartBigIcon, SailboatIcon } from "lucide-react";
import defaultMdxComponents from "fumadocs-ui/mdx";
export default function PresetsBuilderSection({
config,
setConfig,
}: {
config: Config;
setConfig: any;
}) {
const { Card, Cards } = defaultMdxComponents;
return (
<>
<h3>Presets</h3>
<Cards className="grid-cols-2">
<Card
className={
config.preset === "ckan-only" &&
config.extensions.length === 0 &&
config.features.length === 0
? selectedCardClasses
: "cursor-pointer"
}
icon={<SailboatIcon />}
title="CKAN-only"
onClick={() => {
setConfig({
...config,
preset: "ckan-only",
extensions: [],
features: [],
});
}}
>
Installs CKAN with ckan-compose. No CKAN extensions and extra features
are installed.
</Card>
<Card
className={
config.preset === "dathere-default"
? selectedCardClasses
: "cursor-pointer"
}
icon={<BarChartBigIcon />}
title="datHere Default"
onClick={() => {
setConfig({
...config,
preset: "dathere-default",
ckanVersion: "2.11.3",
extensions: ["ckanext-scheming", "DataStore", "DataPusher+"],
features: ["enable-ssh"],
});
}}
>
datHere's default preset featuring the DataPusher+ extension.
</Card>
</Cards>
</>
);
}

View file

@ -8,49 +8,23 @@ import {
SailboatIcon,
TerminalSquareIcon,
} from "lucide-react";
import { useEffect, useState } from "react";
import PresetsBuilderSection from "./builder-sections/presets";
import CKANVersionBuilderSection from "./builder-sections/ckan-version";
import CKANExtensionsBuilderSection from "./builder-sections/ckan-extensions";
import FeaturesBuilderSection from "./builder-sections/features";
import { useState } from "react";
export type Config = {
type Config = {
preset: string | undefined;
ckanVersion: string;
extensions: string[];
features: string[];
};
export const selectedCardClasses =
"bg-blue-100 dark:bg-blue-950 border-blue-300 dark:border-blue-900 border-2";
export default function Builder() {
const { Card, Cards } = defaultMdxComponents;
const [command, setCommand] = useState("./ckan-devstaller");
const [config, setConfig] = useState<Config>({
preset: "ckan-only",
preset: undefined,
ckanVersion: "2.11.3",
extensions: [],
features: [],
});
// Update command string when user changes configuration
useEffect(() => {
const ckanVersionString = `--ckan-version ${config.ckanVersion}`;
const extensionsString =
config.extensions.length > 0
? ` \\\n--extensions ${config.extensions.join(" ")}`
: undefined;
const featuresString =
config.features.length > 0
? ` \\\n--features ${config.features.join(" ")}`
: undefined;
setCommand(
`./ckan-devstaller \\
${ckanVersionString}${extensionsString ? extensionsString : ""}${featuresString ? featuresString : ""}`,
);
}, [config]);
return (
<div className="grid grid-cols-3 gap-4">
<div className="col-span-1 border-r-2 pr-4">
@ -61,38 +35,72 @@ ${ckanVersionString}${extensionsString ? extensionsString : ""}${featuresString
</CodeBlock>
<h2>Selected configuration</h2>
<div>
<strong>CKAN version</strong>: {config.ckanVersion}
<strong>CKAN version</strong>: 2.11.3
<br />
<br />
{config.extensions.length > 0 && (
<>
<strong>Extensions:</strong>
<ul>
{config.extensions.map((extension) => (
<li key={extension}>{extension}</li>
))}
<li>DataStore</li>
<li>ckanext-scheming</li>
<li>DataPusher+</li>
</ul>
</>
)}
{config.features.length > 0 && (
<>
<strong>Features:</strong>
<strong>Extra features:</strong>
<ul>
{config.features.map((feature) => (
<li key={feature}>{feature}</li>
))}
<li>Enable SSH</li>
</ul>
</>
)}
</div>
</div>
</div>
<div className="col-span-2">
<h2>Configuration options</h2>
<PresetsBuilderSection config={config} setConfig={setConfig} />
<CKANVersionBuilderSection config={config} setConfig={setConfig} />
<CKANExtensionsBuilderSection config={config} setConfig={setConfig} />
<FeaturesBuilderSection config={config} setConfig={setConfig} />
<h3>Presets</h3>
<Cards className="grid-cols-2">
<Card
className="bg-blue-100 dark:bg-blue-950 border-blue-300 dark:border-blue-900 border-2"
icon={<SailboatIcon />}
title="CKAN-only"
>
Installs CKAN with ckan-compose.
</Card>
<Card icon={<BlocksIcon />} title="CKAN and the DataStore extension">
Installs CKAN and the DataStore extension.
</Card>
<Card icon={<BarChartBigIcon />} title="datHere Default">
Installs CKAN, the DataStore extension, the ckanext-scheming
extension, and the DataPusher+ extension.
</Card>
</Cards>
<h3>CKAN version</h3>
<Cards>
<Card icon={<SailboatIcon />} title="2.11.3"></Card>
<Card icon={<SailboatIcon />} title="2.10.8"></Card>
<Card
icon={<SailboatIcon />}
title="Install a different version"
></Card>
<Card
icon={<SailboatIcon />}
title="Clone from remote Git repository"
></Card>
</Cards>
<h3>CKAN extensions</h3>
<Cards>
<Card icon={<TerminalSquareIcon />} title="ckanext-scheming"></Card>
<Card icon={<TerminalSquareIcon />} title="ckanext-gztr"></Card>
<Card icon={<TerminalSquareIcon />} title="DataStore"></Card>
<Card icon={<TerminalSquareIcon />} title="DataPusher+"></Card>
<Card icon={<TerminalSquareIcon />} title="ckanext-spatial"></Card>
<Card icon={<TerminalSquareIcon />} title="Custom extension"></Card>
</Cards>
<h3>Extra features</h3>
<Cards>
<Card icon={<TerminalSquareIcon />} title="Enable SSH">
Installs openssh-server and net-tools.
</Card>
<Card icon={<TerminalSquareIcon />} title="Run a Bash script">
Run a Bash script before or after any step during the installation.
</Card>
</Cards>
</div>
</div>
);

View file

@ -20,7 +20,6 @@ const buttonVariants = cva(
sm: "h-9 px-3",
lg: "h-11 px-6",
xs: "px-2 py-1.5 text-xs",
"icon-xs": "p-1 [&_svg]:size-4"
},
},
defaultVariants: {

View file

@ -1,40 +0,0 @@
"use client";
import {
CircleCheckIcon,
InfoIcon,
Loader2Icon,
OctagonXIcon,
TriangleAlertIcon,
} from "lucide-react";
import { useTheme } from "next-themes";
import { Toaster as Sonner, ToasterProps } from "sonner";
const Toaster = ({ ...props }: ToasterProps) => {
const { theme = "system" } = useTheme();
return (
<Sonner
theme={theme as ToasterProps["theme"]}
className="toaster group"
icons={{
success: <CircleCheckIcon className="size-4" />,
info: <InfoIcon className="size-4" />,
warning: <TriangleAlertIcon className="size-4" />,
error: <OctagonXIcon className="size-4" />,
loading: <Loader2Icon className="size-4 animate-spin" />,
}}
style={
{
"--normal-bg": "var(--popover)",
"--normal-text": "var(--popover-foreground)",
"--normal-border": "var(--border)",
"--border-radius": "var(--radius)",
} as React.CSSProperties
}
{...props}
/>
);
};
export { Toaster };

View file

@ -1,37 +0,0 @@
---
title: Changelog for ckan-devstaller v0.3.0 (2025-10-14)
---
Since v0.2.1 of ckan-devstaller, there have been many new features and changes now available in v0.3.0.
## New web app: ckan-devstaller.dathere.com
We've released a new web app [ckan-devstaller.dathere.com](https://ckan-devstaller.dathere.com) as the primary documentation site for ckan-devstaller.
## Builder page
There is now an interactive web GUI, the [Builder](/docs/builder), for users to customize their CKAN installation before copying the (now updated) ckan-devstaller command and running it on their terminal. This helps resolve issue [#6](https://github.com/dathere/ckan-devstaller/issues/11).
## Updated Quick Start page
The [Quick Start](/docs) page now includes three options for suggested installation methods:
1. Use the interactive [Builder](/docs/builder).
2. Install the "CKAN-only" preset with a script which installs the latest stable version of CKAN and ckan-compose (with optional non-interactive script).
3. Install the "datHere default" preset with a script which installs the latest stable version of CKAN and ckan-compose (with optional non-interactive script) along with the DataStore, ckanext-scheming, and DataPusher+ extensions and also installs the `openssh-server` package.
## Installation architecture page
There is now an [Installation Architecture](/docs/reference/installation-architecture) page in the Reference section of the web app that provides a visual representaion of where `ckan-devstaller` installs relevant files/folders.
## Uninstall CKAN page
There is now an [Uninstall CKAN](/docs/tutorials/uninstall-ckan) page in the Tutorials section of the web app that helps users understand how to uninstall their newly installed CKAN installation. This includes the option to either use the new `ckan-devstaller uninstall` subcommand or run the script directly.
## README update
The README on the [ckan-devstaller GitHub repository](https://github.com/dathere/ckan-devstaller) has been updated to have a more user-friendly focus for users and developers that may be new to CKAN thanks to the suggestions by [@drw](https://github.com/drw) in issues [#10](https://github.com/dathere/ckan-devstaller/issues/10) and [#11](https://github.com/dathere/ckan-devstaller/issues/11).
## Changelog section
We've added a Changelog section to the web app to denote new changes to `ckan-devstaller` for each release.

View file

@ -15,56 +15,28 @@ import { Accordion, Accordions } from 'fumadocs-ui/components/accordion';
<Accordion title="Note for ARM64 users">Currently `ckan-devstaller` supports `x86_64` architecture. `ARM64` support is planned.</Accordion>
</Accordions>
---
Currently you have two options to choose from for installation. Paste one of the following scripts into your new Ubuntu 22.04 instance's terminal:
You have several options to choose from for installation. Here are a few:
## [1/3] Customize your CKAN installation with the Builder (Recommended)
<Card
icon={<BlocksIcon />}
href="/docs/builder"
title="Builder"
>
Click here to customize your CKAN installation with an interactive web GUI
</Card>
## [2/3] Install the "CKAN-only" preset
By running the following script, ckan-devstaller will be downloaded and the default configuration for installing CKAN 2.11.3 with ckan-compose will be selected. You can then customize your configuration interactively in your terminal after running this script.
## (Option 1/2) Install with interactive mode
```bash
wget -O - https://github.com/dathere/ckan-devstaller/releases/download/0.3.0/install.bash | bash
wget -O - https://github.com/dathere/ckan-devstaller/releases/download/0.2.1/install.bash | bash -s default
```
If you'd rather skip the interactivity and go straight to installation, then run the following script instead:
## (Option 2/2) Install with non-interactive mode with a specific config
```bash
wget -O - https://github.com/dathere/ckan-devstaller/releases/download/0.3.0/install.bash | bash -s skip-interactive
```
## [3/3] Install the "datHere Default" preset
The following script will download ckan-devstaller and select the following configuration:
The following script will install the following:
- CKAN 2.11.3
- [ckan-compose](https://github.com/tino097/ckan-compose/tree/ckan-devstaller)
- [DataStore extension](https://docs.ckan.org/en/2.11/maintaining/datastore.html)
- [ckanext-scheming extension](https://github.com/ckan/ckanext-scheming)
- [DataPusher+ extension](https://github.com/dathere/datapusher-plus)
- Install the `openssh-server` package for allowing SSH capability
- [DRUF mode](https://github.com/dathere/datapusher-plus?tab=readme-ov-file#druf-dataset-resource-upload-first-workflow) for DataPusher+ is available but disabled by default.
You can then customize your configuration interactively in your terminal after running this script.
[DRUF mode](https://github.com/dathere/datapusher-plus?tab=readme-ov-file#druf-dataset-resource-upload-first-workflow) for DataPusher+ is available but disabled by default.
```bash
wget -O - https://github.com/dathere/ckan-devstaller/releases/download/0.3.0/install.bash | bash -s dathere-default
```
If you'd rather skip the interactivity and go straight to installation, then run the following script instead:
```bash
wget -O - https://github.com/dathere/ckan-devstaller/releases/download/0.3.0/install.bash | bash -s dathere-default skip-interactive
wget -O - https://github.com/dathere/ckan-devstaller/releases/download/0.2.1/install.bash | bash -s default
```
## Learn more

View file

@ -1,11 +1,3 @@
{
"pages": [
"---Introduction---",
"index",
"builder",
"---Further reading---",
"tutorials",
"reference",
"changelog"
]
"pages": ["---Introduction---", "index", "what-is-ckan-devstaller", "builder", "---Further reading---", "tutorials", "reference"]
}

View file

@ -5,13 +5,7 @@ description: How to uninstall CKAN after having installed with ckan-devstaller
You may want to uninstall CKAN and related files after having ran ckan-devstaller. This can be useful if you want to re-run ckan-devstaller with a different configuration or are developing ckan-devstaller.
The uninstallation process can be done by running:
```bash
./ckan-devstaller uninstall
```
The following script will be ran to uninstall CKAN and files related to ckan-devstaller:
Run the following script to uninstall CKAN and files related to ckan-devstaller:
```bash
sudo rm -rf /usr/lib/ckan

View file

@ -1,6 +0,0 @@
import { clsx, type ClassValue } from "clsx"
import { twMerge } from "tailwind-merge"
export function cn(...inputs: ClassValue[]) {
return twMerge(clsx(inputs))
}

View file

@ -15,20 +15,17 @@
"@radix-ui/react-collapsible": "^1.1.12",
"@radix-ui/react-tabs": "^1.1.13",
"class-variance-authority": "^0.7.1",
"clsx": "^2.1.1",
"fumadocs-core": "15.8.1",
"fumadocs-mdx": "12.0.1",
"fumadocs-ui": "15.8.1",
"lucide-react": "^0.545.0",
"lucide-react": "^0.544.0",
"next": "15.5.4",
"next-themes": "^0.4.6",
"react": "^19.1.1",
"react-dom": "^19.1.1",
"sonner": "^2.0.7",
"tailwind-merge": "^3.3.1"
},
"devDependencies": {
"@biomejs/biome": "2.2.6",
"@biomejs/biome": "2.2.5",
"@tailwindcss/postcss": "^4.1.13",
"@types/mdx": "^2.0.13",
"@types/node": "24.5.2",
@ -36,7 +33,6 @@
"@types/react-dom": "^19.1.9",
"postcss": "^8.5.6",
"tailwindcss": "^4.1.13",
"tw-animate-css": "^1.4.0",
"typescript": "^5.9.2"
}
}

View file

@ -11,25 +11,18 @@ sudo apt install curl -y
cd ~/
# Download the ckan-devstaller binary file
curl -LO https://github.com/dathere/ckan-devstaller/releases/download/0.3.0/ckan-devstaller
curl -LO https://github.com/dathere/ckan-devstaller/releases/download/0.2.1/ckan-devstaller
# Add execute permission to ckan-devstaller binary file
sudo chmod +x ./ckan-devstaller
# Run the ckan-devstaller binary file with the specified preset and (non-)interactive mode
preset=$1
skip_interactive=$2
# Run the ckan-devstaller binary file
# If the user provides an argument "default", run ckan-devstaller in non-interactive mode with the default config
# Otherwise run ckan-devstaller in interactive mode
flag=$1
if [ $preset == "dathere-default" ]; then
if [ $skip_interactive == "skip-interactive" ]; then
./ckan-devstaller --ckan-version 2.11.3 --extensions ckanext-scheming DataStore DataPusher+ --features enable-ssh --skip-interactive
else
./ckan-devstaller --ckan-version 2.11.3 --extensions ckanext-scheming DataStore DataPusher+ --features enable-ssh
fi
if [ $flag == "default" ]; then
./ckan-devstaller --default
else
if [ $preset == "skip-interactive" ]; then
./ckan-devstaller --skip-interactive
else
./ckan-devstaller
fi
fi

View file

@ -4,49 +4,26 @@ mod styles;
use crate::{
questions::{question_ckan_version, question_ssh, question_sysadmin},
steps::{
step_install_ahoy, step_install_and_run_ckan_compose,
step_install_ckanext_scheming_extension, step_install_curl,
step_install_datapusher_plus_extension, step_install_datastore_extension,
step_install_docker, step_install_openssh, step_package_updates,
},
styles::{important_text, step_text, success_text},
styles::{highlighted_text, important_text, step_text, success_text},
};
use anyhow::Result;
use clap::{Parser, Subcommand};
use clap::Parser;
use human_panic::{metadata, setup_panic};
use inquire::Confirm;
use serde_json::json;
use std::{path::PathBuf, str::FromStr};
use xshell::cmd;
use xshell_venv::{Shell, VirtualEnv};
/// CLI to help install a CKAN instance for development within minutes. Learn more at: https://ckan-devstaller.dathere.com
#[derive(Parser)]
/// ckan-devstaller CLI
#[derive(Parser, Debug)]
#[command(version, about, long_about = None)]
struct Args {
/// Skip interactive steps
/// Skip interactive steps and install CKAN with default features
#[arg(short, long)]
skip_interactive: bool,
#[arg(short, long)]
/// CKAN version to install defined by semantic versioning from official releases from https://github.com/ckan/ckan
ckan_version: Option<String>,
/// List of CKAN extensions to install, separated by spaces
#[arg(short, long, value_parser, num_args = 1.., value_delimiter = ' ')]
extensions: Option<Vec<String>>,
/// List of custom features, separated by spaces
#[arg(short, long, value_parser, num_args = 1.., value_delimiter = ' ')]
features: Option<Vec<String>>,
#[command(subcommand)]
command: Option<Commands>,
default: bool,
}
#[derive(Subcommand)]
enum Commands {
/// Attempt to uninstall CKAN and related ckan-devstaller installation files
Uninstall {},
}
#[derive(Clone)]
struct Sysadmin {
username: String,
password: String,
@ -68,91 +45,37 @@ fn main() -> Result<()> {
.homepage("https://dathere.com")
.support("- Create a support ticket at https://support.dathere.com or report an issue at https://github.com/dathere/ckan-devstaller"));
// Set up default config
let args = Args::parse();
// Set up default config
let sh = Shell::new()?;
let username = cmd!(sh, "whoami").read()?;
steps::step_intro();
if matches!(&args.command, Some(Commands::Uninstall {})) {
let uninstall_confirmation = Confirm::new(
"Are you sure you want to uninstall CKAN and related files from ckan-devstaller?",
)
.with_help_message(
r#"The following commands are ran when attempting the uninstall:
sudo rm -rf /usr/lib/ckan
sudo rm -rf /etc/ckan
cd ~/
rm -rf qsv*
rm -rf README ckan-compose ahoy dpp_default_config.ini get-docker.sh permissions.sql"#,
)
.prompt()?;
if uninstall_confirmation {
cmd!(sh, "sudo rm -rf /usr/lib/ckan").run()?;
cmd!(sh, "sudo rm -rf /etc/ckan").run()?;
sh.change_dir(format!("/home/{username}"));
cmd!(sh, "rm -rf qsv*").run()?;
cmd!(sh, "rm -rf README ckan-compose ahoy dpp_default_config.ini get-docker.sh permissions.sql").run()?;
let default_config_text = r#"
The default configuration for ckan-devstaller does the following:
- Install openssh-server to enable SSH access
- Install ckan-compose (https://github.com/tino097/ckan-compose) which sets up the CKAN backend (PostgreSQL, SOLR, Redis)
- Install CKAN v2.11.3
- Install the DataStore extension
- Install the ckanext-scheming extension
- Install the DataPusher+ extension
- Disable DRUF mode for DataPusher+
"#;
println!("{default_config_text}");
let answer_customize = if args.default {
false
} else {
println!("Cancelling command.");
}
return Ok(());
}
Confirm::new(
"Would you like to customize any of these features for your CKAN installation?",
)
.prompt()?
};
let default_sysadmin = Sysadmin {
username: username.clone(),
password: "password".to_string(),
email: format!("{username}@localhost"),
};
let config = Config {
ssh: args
.features
.is_some_and(|features| features.contains(&"enable-ssh".to_string())),
ckan_version: if args.ckan_version.is_some() {
args.ckan_version.unwrap()
} else {
"2.11.3".to_string()
},
sysadmin: default_sysadmin.clone(),
extension_datastore: args
.extensions
.clone()
.is_some_and(|extensions| extensions.contains(&"DataStore".to_string())),
extension_ckanext_scheming: args
.extensions
.clone()
.is_some_and(|extensions| extensions.contains(&"ckanext-scheming".to_string())),
extension_datapusher_plus: args
.extensions
.is_some_and(|extensions| extensions.contains(&"DataPusher+".to_string())),
druf_mode: false,
};
steps::step_intro();
let mut default_config_text =
String::from("The current configuration for ckan-devstaller does the following:");
if config.ssh {
default_config_text.push_str("\n- Install openssh-server to enable SSH access");
}
default_config_text.push_str("\n- Install ckan-compose (https://github.com/tino097/ckan-compose) which sets up the CKAN backend (PostgreSQL, SOLR, Redis)");
default_config_text.push_str(format!("\n- Install CKAN v{}", config.ckan_version).as_str());
if config.extension_datastore {
default_config_text.push_str("\n- Install the DataStore extension");
}
if config.extension_ckanext_scheming {
default_config_text.push_str("\n- Install the ckanext-scheming extension");
}
if config.extension_datapusher_plus {
default_config_text.push_str("\n- Install the DataPusher+ extension");
default_config_text.push_str("\n- Disable DRUF mode for DataPusher+");
}
println!("{default_config_text}");
let answer_customize = if args.skip_interactive {
false
} else {
Confirm::new("Would you like to customize the configuration for your CKAN installation?")
.prompt()?
};
let config = if answer_customize {
let answer_ssh = question_ssh()?;
let answer_ckan_version = question_ckan_version()?;
@ -184,33 +107,109 @@ rm -rf README ckan-compose ahoy dpp_default_config.ini get-docker.sh permissions
druf_mode: answer_druf_mode,
}
} else {
config
Config {
ssh: true,
ckan_version: "2.11.3".to_string(),
sysadmin: default_sysadmin,
extension_datastore: true,
extension_ckanext_scheming: true,
extension_datapusher_plus: true,
druf_mode: false,
}
};
let begin_installation = if args.skip_interactive {
let begin_installation = if args.default {
true
} else {
Confirm::new("Would you like to begin the installation?").prompt()?
};
if begin_installation {
println!("\n{}", important_text("Starting installation..."));
// Run sudo apt update and sudo apt upgrade
step_package_updates("1.".to_string(), &sh)?;
println!("{}", important_text("Starting installation..."));
println!(
"\n{} Running {} and {}...",
step_text("1."),
highlighted_text("sudo apt update -y"),
highlighted_text("sudo apt upgrade -y")
);
println!(
"{}",
important_text("You may need to provide your sudo password.")
);
cmd!(sh, "sudo apt update -y").run()?;
// Ignoring xrdp error with .ignore_status() for now
cmd!(sh, "sudo apt upgrade -y").ignore_status().run()?;
println!(
"{}",
success_text("✅ 1. Successfully ran update and upgrade commands.")
);
// Install curl
step_install_curl("2.".to_string(), &sh)?;
// If user wants SSH capability, install openssh-server
println!(
"\n{} Installing {}...",
step_text("2."),
highlighted_text("curl")
);
cmd!(sh, "sudo apt install curl -y").run()?;
println!("{}", success_text("✅ 2.1. Successfully installed curl."));
if config.ssh {
step_install_openssh("2.".to_string(), &sh)?;
println!("\n{} Installing openssh-server...", step_text("2."));
cmd!(sh, "sudo apt install openssh-server -y").run()?;
}
println!(
"{}",
success_text("✅ 2.2. Successfully installed openssh-server.")
);
let dpkg_l_output = cmd!(sh, "dpkg -l").read()?;
let has_docker = cmd!(sh, "grep docker")
.stdin(dpkg_l_output.clone())
.ignore_status()
.output()?
.status
.success();
if !has_docker {
println!("{} Installing Docker...", step_text("3."),);
cmd!(
sh,
"curl -fsSL https://get.docker.com -o /home/{username}/get-docker.sh"
)
.run()?;
cmd!(sh, "sudo sh /home/{username}/get-docker.sh").run()?;
println!("{}", success_text("✅ 3. Successfully installed Docker."));
}
// Install docker CLI if user does not have it installed
step_install_docker("3.".to_string(), &sh, username.clone())?;
let has_docker_compose = cmd!(sh, "grep docker-compose")
.stdin(dpkg_l_output)
.ignore_status()
.output()?
.status
.success();
if !has_docker_compose {
cmd!(sh, "sudo apt install docker-compose -y").run()?;
}
step_install_ahoy("4.".to_string(), &sh, username.clone())?;
println!("\n{} Installing Ahoy...", step_text("4."),);
sh.change_dir(format!("/home/{username}"));
cmd!(sh, "sudo curl -LO https://github.com/ahoy-cli/ahoy/releases/download/v2.5.0/ahoy-bin-linux-amd64").run()?;
cmd!(sh, "mv ./ahoy-bin-linux-amd64 ./ahoy").run()?;
cmd!(sh, "sudo chmod +x ./ahoy").run()?;
println!("{}", success_text("✅ 4. Successfully installed Ahoy."));
step_install_and_run_ckan_compose("5.".to_string(), &sh, username.clone())?;
println!(
"\n{} Downloading, installing, and starting ckan-compose...",
step_text("5."),
);
if !std::fs::exists(format!("/home/{username}/ckan-compose"))? {
cmd!(sh, "git clone https://github.com/tino097/ckan-compose.git").run()?;
}
sh.change_dir(format!("/home/{username}/ckan-compose"));
cmd!(sh, "git switch ckan-devstaller").run()?;
let env_data = "PROJECT_NAME=ckan-devstaller-project
DATASTORE_READONLY_PASSWORD=pass
POSTGRES_PASSWORD=pass";
std::fs::write(format!("/home/{username}/ckan-compose/.env"), env_data)?;
cmd!(sh, "sudo ../ahoy up").run()?;
println!("{}", success_text("✅ 5. Successfully ran ckan-compose."));
println!(
"\n{} Installing CKAN {}...",
@ -261,29 +260,210 @@ rm -rf README ckan-compose ahoy dpp_default_config.ini get-docker.sh permissions
.run()?;
println!(
"{}",
success_text(format!("6. Installed CKAN {}.", config.ckan_version).as_str())
success_text(format!("6. Installed CKAN {}.", config.ckan_version).as_str())
);
// Install extensions
if config.extension_datastore {
step_install_datastore_extension("7.".to_string(), &sh, username.clone())?;
}
if config.extension_ckanext_scheming {
step_install_ckanext_scheming_extension("8.".to_string(), &sh, username.clone())?;
}
if config.extension_datapusher_plus {
step_install_datapusher_plus_extension(
"9.".to_string(),
&sh,
sysadmin_username,
username.clone(),
println!(
"\n{} Enabling DataStore plugin, adding config URLs in /etc/ckan/default/ckan.ini and updating permissions...",
step_text("7."),
);
let mut conf = ini::Ini::load_from_file("/etc/ckan/default/ckan.ini")?;
let app_main_section = conf.section_mut(Some("app:main")).unwrap();
let mut ckan_plugins = app_main_section.get("ckan.plugins").unwrap().to_string();
ckan_plugins.push_str(" datastore");
app_main_section.insert("ckan.plugins", ckan_plugins);
app_main_section.insert(
"ckan.datastore.write_url",
"postgresql://ckan_default:pass@localhost/datastore_default",
);
app_main_section.insert(
"ckan.datastore.read_url",
"postgresql://datastore_default:pass@localhost/datastore_default",
);
app_main_section.insert("ckan.datastore.sqlsearch.enabled", "true");
conf.write_to_file("/etc/ckan/default/ckan.ini")?;
let postgres_container_id = cmd!(
sh,
"sudo docker ps -aqf name=^ckan-devstaller-project-postgres$"
)
.read()?;
let set_permissions_output = cmd!(
sh,
"ckan -c /etc/ckan/default/ckan.ini datastore set-permissions"
)
.read()?;
std::fs::write("permissions.sql", set_permissions_output)?;
loop {
std::thread::sleep(std::time::Duration::from_secs(2));
if std::fs::exists("permissions.sql")? {
break;
}
}
sh.change_dir(format!("/home/{username}"));
cmd!(
sh,
"sudo docker cp permissions.sql {postgres_container_id}:/permissions.sql"
)
.run()?;
cmd!(sh, "sudo docker exec {postgres_container_id} psql -U ckan_default --set ON_ERROR_STOP=1 -f permissions.sql").run()?;
println!(
"{}",
success_text(
"✅ 7. Enabled DataStore plugin, set DataStore URLs in /etc/ckan/default/ckan.ini, and updated permissions."
)
);
println!(
"{}",
step_text("\n{} Installing ckanext-scheming and DataPusher+ extensions..."),
);
cmd!(
sh,
"pip install -e git+https://github.com/ckan/ckanext-scheming.git#egg=ckanext-scheming"
)
.run()?;
let mut conf = ini::Ini::load_from_file("/etc/ckan/default/ckan.ini")?;
let app_main_section = conf.section_mut(Some("app:main")).unwrap();
let mut ckan_plugins = app_main_section.get("ckan.plugins").unwrap().to_string();
ckan_plugins.push_str(" scheming_datasets");
cmd!(
sh,
"ckan config-tool /etc/ckan/default/ckan.ini -s app:main ckan.plugins={ckan_plugins}"
)
.run()?;
cmd!(sh, "ckan config-tool /etc/ckan/default/ckan.ini -s app:main scheming.presets=ckanext.scheming:presets.json").run()?;
cmd!(sh, "ckan config-tool /etc/ckan/default/ckan.ini -s app:main scheming.dataset_fallback=false").run()?;
// app_main_section.insert("ckan.plugins", ckan_plugins);
// app_main_section.insert("scheming.presets", "ckanext.scheming:presets.json");
// app_main_section.insert("scheming.dataset_fallback", "false");
// conf.write_to_file("/etc/ckan/default/ckan.ini")?;
// Install DataPusher+
cmd!(sh, "sudo apt install python3-virtualenv python3-dev python3-pip python3-wheel build-essential libxslt1-dev libxml2-dev zlib1g-dev git libffi-dev libpq-dev uchardet -y").run()?;
sh.change_dir("/usr/lib/ckan/default/src");
cmd!(sh, "pip install -e git+https://github.com/dathere/datapusher-plus.git@main#egg=datapusher-plus").run()?;
sh.change_dir("/usr/lib/ckan/default/src/datapusher-plus");
cmd!(sh, "pip install -r requirements.txt").run()?;
sh.change_dir(format!("/home/{username}"));
cmd!(sh, "wget https://github.com/dathere/qsv/releases/download/4.0.0/qsv-4.0.0-x86_64-unknown-linux-gnu.zip").run()?;
cmd!(sh, "sudo apt install unzip -y").run()?;
cmd!(sh, "unzip qsv-4.0.0-x86_64-unknown-linux-gnu.zip").run()?;
cmd!(sh, "sudo rm -rf qsv-4.0.0-x86_64-unknown-linux-gnu.zip").run()?;
cmd!(sh, "sudo mv ./qsvdp_glibc-2.31 /usr/local/bin/qsvdp").run()?;
let mut conf = ini::Ini::load_from_file("/etc/ckan/default/ckan.ini")?;
let app_main_section = conf.section_mut(Some("app:main")).unwrap();
let mut ckan_plugins = app_main_section.get("ckan.plugins").unwrap().to_string();
ckan_plugins.push_str(" datapusher_plus");
cmd!(
sh,
"ckan config-tool /etc/ckan/default/ckan.ini -s app:main ckan.plugins={ckan_plugins}"
)
.run()?;
cmd!(sh, "ckan config-tool /etc/ckan/default/ckan.ini -s app:main scheming.dataset_schemas=ckanext.datapusher_plus:dataset-druf.yaml").run()?;
// app_main_section.insert("ckan.plugins", ckan_plugins);
// app_main_section.insert(
// "scheming.dataset_schemas",
// "ckanext.datapusher_plus:dataset-druf.yaml",
// );
// conf.write_to_file("/etc/ckan/default/ckan.ini")?;
let dpp_default_config = r#"
ckanext.datapusher_plus.use_proxy = false
ckanext.datapusher_plus.download_proxy =
ckanext.datapusher_plus.ssl_verify = false
# supports INFO, DEBUG, TRACE - use DEBUG or TRACE when debugging scheming Formulas
ckanext.datapusher_plus.upload_log_level = INFO
ckanext.datapusher_plus.formats = csv tsv tab ssv xls xlsx xlsxb xlsm ods geojson shp qgis zip
ckanext.datapusher_plus.pii_screening = false
ckanext.datapusher_plus.pii_found_abort = false
ckanext.datapusher_plus.pii_regex_resource_id_or_alias =
ckanext.datapusher_plus.pii_show_candidates = false
ckanext.datapusher_plus.pii_quick_screen = false
ckanext.datapusher_plus.qsv_bin = /usr/local/bin/qsvdp
ckanext.datapusher_plus.preview_rows = 100
ckanext.datapusher_plus.download_timeout = 300
ckanext.datapusher_plus.max_content_length = 1256000000000
ckanext.datapusher_plus.chunk_size = 16384
ckanext.datapusher_plus.default_excel_sheet = 0
ckanext.datapusher_plus.sort_and_dupe_check = true
ckanext.datapusher_plus.dedup = false
ckanext.datapusher_plus.unsafe_prefix = unsafe_
ckanext.datapusher_plus.reserved_colnames = _id
ckanext.datapusher_plus.prefer_dmy = false
ckanext.datapusher_plus.ignore_file_hash = true
ckanext.datapusher_plus.auto_index_threshold = 3
ckanext.datapusher_plus.auto_index_dates = true
ckanext.datapusher_plus.auto_unique_index = true
ckanext.datapusher_plus.summary_stats_options =
ckanext.datapusher_plus.add_summary_stats_resource = false
ckanext.datapusher_plus.summary_stats_with_preview = false
ckanext.datapusher_plus.qsv_stats_string_max_length = 32767
ckanext.datapusher_plus.qsv_dates_whitelist = date,time,due,open,close,created
ckanext.datapusher_plus.qsv_freq_limit = 10
ckanext.datapusher_plus.auto_alias = true
ckanext.datapusher_plus.auto_alias_unique = false
ckanext.datapusher_plus.copy_readbuffer_size = 1048576
ckanext.datapusher_plus.type_mapping = {"String": "text", "Integer": "numeric","Float": "numeric","DateTime": "timestamp","Date": "date","NULL": "text"}
ckanext.datapusher_plus.auto_spatial_simplication = true
ckanext.datapusher_plus.spatial_simplication_relative_tolerance = 0.1
ckanext.datapusher_plus.latitude_fields = latitude,lat
ckanext.datapusher_plus.longitude_fields = longitude,long,lon
ckanext.datapusher_plus.jinja2_bytecode_cache_dir = /tmp/jinja2_butecode_cache
ckanext.datapusher_plus.auto_unzip_one_file = true
ckanext.datapusher_plus.api_token = <CKAN service account token for CKAN user with sysadmin privileges>
ckanext.datapusher_plus.describeGPT_api_key = <Token for OpenAI API compatible service>
ckanext.datapusher_plus.file_bin = /usr/bin/file
ckanext.datapusher_plus.enable_druf = false
ckanext.datapusher_plus.enable_form_redirect = true
"#;
std::fs::write("dpp_default_config.ini", dpp_default_config)?;
cmd!(
sh,
"ckan config-tool /etc/ckan/default/ckan.ini -f dpp_default_config.ini"
)
.run()?;
let resource_formats_str = std::fs::read_to_string(
"/usr/lib/ckan/default/src/ckan/config/resource_formats.json",
)?;
let mut resource_formats_val: serde_json::Value =
serde_json::from_str(&resource_formats_str)?;
let all_resource_formats = resource_formats_val
.get_mut(0)
.unwrap()
.as_array_mut()
.unwrap();
all_resource_formats.push(json!([
"TAB",
"Tab Separated Values File",
"text/tab-separated-values",
[]
]));
std::fs::write(
"/usr/lib/ckan/default/src/ckan/config/resource_formats.json",
serde_json::to_string(&resource_formats_val)?,
)?;
cmd!(sh, "sudo locale-gen en_US.UTF-8").run()?;
cmd!(sh, "sudo update-locale").run()?;
let token_command_output = cmd!(
sh,
"ckan -c /etc/ckan/default/ckan.ini user token add {sysadmin_username} dpplus"
)
.read()?;
let tail_output = cmd!(sh, "tail -n 1").stdin(token_command_output).read()?;
let dpp_api_token = cmd!(sh, "tr -d '\t'").stdin(tail_output).read()?;
cmd!(sh, "ckan config-tool /etc/ckan/default/ckan.ini ckanext.datapusher_plus.api_token={dpp_api_token}").env("LC_ALL", "en_US.UTF-8").run()?;
cmd!(
sh,
"ckan -c /etc/ckan/default/ckan.ini db upgrade -p datapusher_plus"
)
.run()?;
println!(
"{}",
success_text("✅ 8. Installed ckanext-scheming and DataPusher+ extensions.")
);
}
println!("\n{}", success_text("Running CKAN instance..."));
println!("\n{}", success_text("Running CKAN instance..."));
cmd!(sh, "ckan -c /etc/ckan/default/ckan.ini run").run()?;
} else {
println!("Cancelling installation.");
}
Ok(())

View file

@ -1,7 +1,4 @@
use crate::styles::{highlighted_text, important_text, step_text, success_text};
use anyhow::Result;
use serde_json::json;
use xshell::{Shell, cmd};
use crate::styles::{highlighted_text, important_text};
pub fn step_intro() {
println!("Welcome to the ckan-devstaller!");
@ -14,374 +11,9 @@ pub fn step_intro() {
highlighted_text("CKAN 2.11.3")
);
println!(
"\nYou may also learn more about ckan-devstaller at https://ckan-devstaller.dathere.com."
);
println!(
"\n{}\n",
"{}",
important_text(
"This installer is only intended for a brand new installation of Ubuntu 22.04."
)
);
}
pub fn step_package_updates(step_prefix: String, sh: &Shell) -> Result<()> {
println!(
"\n{} Running {} and {}...",
step_text(step_prefix.as_str()),
highlighted_text("sudo apt update -y"),
highlighted_text("sudo apt upgrade -y")
);
println!(
"{}",
important_text("You may need to provide your sudo password.")
);
cmd!(sh, "sudo apt update -y").run()?;
// Ignoring xrdp error with .ignore_status() for now
cmd!(sh, "sudo apt upgrade -y").ignore_status().run()?;
println!(
"{}",
success_text(
format!("{step_prefix} Successfully ran update and upgrade commands.").as_str()
)
);
Ok(())
}
pub fn step_install_curl(step_prefix: String, sh: &Shell) -> Result<()> {
println!(
"\n{} Installing {}...",
step_text("2."),
highlighted_text("curl")
);
cmd!(sh, "sudo apt install curl -y").run()?;
println!(
"{}",
success_text(format!("{step_prefix} Successfully installed curl.").as_str())
);
Ok(())
}
pub fn step_install_openssh(step_prefix: String, sh: &Shell) -> Result<()> {
println!(
"\n{} Installing openssh-server...",
step_text(step_prefix.as_str())
);
cmd!(sh, "sudo apt install openssh-server -y").run()?;
println!(
"{}",
success_text(format!("{step_prefix} Successfully installed openssh-server.").as_str())
);
Ok(())
}
pub fn step_install_docker(step_prefix: String, sh: &Shell, username: String) -> Result<()> {
let dpkg_l_output = cmd!(sh, "dpkg -l").read()?;
let has_docker = cmd!(sh, "grep docker")
.stdin(dpkg_l_output.clone())
.ignore_status()
.output()?
.status
.success();
if !has_docker {
println!("{} Installing Docker...", step_text(step_prefix.as_str()),);
cmd!(
sh,
"curl -fsSL https://get.docker.com -o /home/{username}/get-docker.sh"
)
.run()?;
cmd!(sh, "sudo sh /home/{username}/get-docker.sh").run()?;
println!(
"{}",
success_text(format!("{step_prefix} Successfully installed Docker.").as_str())
);
}
Ok(())
}
pub fn step_install_ahoy(step_prefix: String, sh: &Shell, username: String) -> Result<()> {
println!("\n{} Installing Ahoy...", step_text(step_prefix.as_str()),);
sh.change_dir(format!("/home/{username}"));
cmd!(sh, "sudo curl -LO https://github.com/ahoy-cli/ahoy/releases/download/v2.5.0/ahoy-bin-linux-amd64").run()?;
cmd!(sh, "mv ./ahoy-bin-linux-amd64 ./ahoy").run()?;
cmd!(sh, "sudo chmod +x ./ahoy").run()?;
println!(
"{}",
success_text(format!("{step_prefix} Successfully installed Ahoy.").as_str())
);
Ok(())
}
pub fn step_install_and_run_ckan_compose(
step_prefix: String,
sh: &Shell,
username: String,
) -> Result<()> {
println!(
"\n{} Downloading, installing, and starting ckan-compose...",
step_text(step_prefix.as_str()),
);
if !std::fs::exists(format!("/home/{username}/ckan-compose"))? {
cmd!(sh, "git clone https://github.com/tino097/ckan-compose.git").run()?;
}
sh.change_dir(format!("/home/{username}/ckan-compose"));
cmd!(sh, "git switch ckan-devstaller").run()?;
let env_data = "PROJECT_NAME=ckan-devstaller-project
DATASTORE_READONLY_PASSWORD=pass
POSTGRES_PASSWORD=pass";
std::fs::write(format!("/home/{username}/ckan-compose/.env"), env_data)?;
cmd!(sh, "sudo ../ahoy up").run()?;
println!(
"{}",
success_text(format!("{step_prefix} Successfully ran ckan-compose.").as_str())
);
Ok(())
}
pub fn step_install_datastore_extension(
step_prefix: String,
sh: &Shell,
username: String,
) -> Result<()> {
println!(
"\n{} Enabling DataStore plugin, adding config URLs in /etc/ckan/default/ckan.ini and updating permissions...",
step_text(step_prefix.as_str()),
);
let mut conf = ini::Ini::load_from_file("/etc/ckan/default/ckan.ini")?;
let app_main_section = conf.section_mut(Some("app:main")).unwrap();
let mut ckan_plugins = app_main_section.get("ckan.plugins").unwrap().to_string();
ckan_plugins.push_str(" datastore");
app_main_section.insert("ckan.plugins", ckan_plugins);
app_main_section.insert(
"ckan.datastore.write_url",
"postgresql://ckan_default:pass@localhost/datastore_default",
);
app_main_section.insert(
"ckan.datastore.read_url",
"postgresql://datastore_default:pass@localhost/datastore_default",
);
app_main_section.insert("ckan.datastore.sqlsearch.enabled", "true");
conf.write_to_file("/etc/ckan/default/ckan.ini")?;
let postgres_container_id = cmd!(
sh,
"sudo docker ps -aqf name=^ckan-devstaller-project-postgres$"
)
.read()?;
let set_permissions_output = cmd!(
sh,
"ckan -c /etc/ckan/default/ckan.ini datastore set-permissions"
)
.read()?;
std::fs::write("permissions.sql", set_permissions_output)?;
loop {
std::thread::sleep(std::time::Duration::from_secs(2));
if std::fs::exists("permissions.sql")? {
break;
}
}
sh.change_dir(format!("/home/{username}"));
cmd!(
sh,
"sudo docker cp permissions.sql {postgres_container_id}:/permissions.sql"
)
.run()?;
cmd!(sh, "sudo docker exec {postgres_container_id} psql -U ckan_default --set ON_ERROR_STOP=1 -f permissions.sql").run()?;
println!(
"{}",
success_text(
format!("{step_prefix} Enabled DataStore plugin, set DataStore URLs in /etc/ckan/default/ckan.ini, and updated permissions.").as_str()
)
);
Ok(())
}
pub fn step_install_ckanext_scheming_extension(
step_prefix: String,
sh: &Shell,
username: String,
) -> Result<()> {
println!(
"{}",
step_text("\n{} Installing the ckanext-scheming extension..."),
);
cmd!(
sh,
"pip install -e git+https://github.com/ckan/ckanext-scheming.git#egg=ckanext-scheming"
)
.run()?;
let mut conf = ini::Ini::load_from_file("/etc/ckan/default/ckan.ini")?;
let app_main_section = conf.section_mut(Some("app:main")).unwrap();
let mut ckan_plugins = app_main_section.get("ckan.plugins").unwrap().to_string();
ckan_plugins.push_str(" scheming_datasets");
cmd!(
sh,
"ckan config-tool /etc/ckan/default/ckan.ini -s app:main ckan.plugins={ckan_plugins}"
)
.run()?;
cmd!(sh, "ckan config-tool /etc/ckan/default/ckan.ini -s app:main scheming.presets=ckanext.scheming:presets.json").run()?;
cmd!(
sh,
"ckan config-tool /etc/ckan/default/ckan.ini -s app:main scheming.dataset_fallback=false"
)
.run()?;
// app_main_section.insert("ckan.plugins", ckan_plugins);
// app_main_section.insert("scheming.presets", "ckanext.scheming:presets.json");
// app_main_section.insert("scheming.dataset_fallback", "false");
// conf.write_to_file("/etc/ckan/default/ckan.ini")?;
Ok(())
}
pub fn step_install_datapusher_plus_extension(
step_prefix: String,
sh: &Shell,
sysadmin_username: String,
username: String,
) -> Result<()> {
// Install DataPusher+
println!(
"{}",
step_text(format!("\n{step_prefix} Installing DataPusher+ extension...").as_str())
);
cmd!(sh, "sudo apt install python3-virtualenv python3-dev python3-pip python3-wheel build-essential libxslt1-dev libxml2-dev zlib1g-dev git libffi-dev libpq-dev uchardet -y").run()?;
sh.change_dir("/usr/lib/ckan/default/src");
cmd!(
sh,
"pip install -e git+https://github.com/dathere/datapusher-plus.git@main#egg=datapusher-plus"
)
.run()?;
sh.change_dir("/usr/lib/ckan/default/src/datapusher-plus");
cmd!(sh, "pip install -r requirements.txt").run()?;
sh.change_dir(format!("/home/{username}"));
cmd!(sh, "wget https://github.com/dathere/qsv/releases/download/4.0.0/qsv-4.0.0-x86_64-unknown-linux-gnu.zip").run()?;
cmd!(sh, "sudo apt install unzip -y").run()?;
cmd!(sh, "unzip qsv-4.0.0-x86_64-unknown-linux-gnu.zip").run()?;
cmd!(sh, "sudo rm -rf qsv-4.0.0-x86_64-unknown-linux-gnu.zip").run()?;
cmd!(sh, "sudo mv ./qsvdp_glibc-2.31 /usr/local/bin/qsvdp").run()?;
let mut conf = ini::Ini::load_from_file("/etc/ckan/default/ckan.ini")?;
let app_main_section = conf.section_mut(Some("app:main")).unwrap();
let mut ckan_plugins = app_main_section.get("ckan.plugins").unwrap().to_string();
ckan_plugins.push_str(" datapusher_plus");
app_main_section.insert("ckan.plugins", ckan_plugins);
app_main_section.insert(
"scheming.dataset_schemas",
"ckanext.datapusher_plus:dataset-druf.yaml",
);
app_main_section.insert("ckanext.datapusher_plus.use_proxy", "false");
app_main_section.insert("ckanext.datapusher_plus.download_proxy", "");
app_main_section.insert("ckanext.datapusher_plus.ssl_verify", "false");
app_main_section.insert("ckanext.datapusher_plus.upload_log_level", "INFO");
app_main_section.insert(
"ckanext.datapusher_plus.formats",
"csv tsv tab ssv xls xlsx xlsxb xlsm ods geojson shp qgis zip",
);
app_main_section.insert("ckanext.datapusher_plus.pii_screening", "false");
app_main_section.insert("ckanext.datapusher_plus.pii_found_abort", "false");
app_main_section.insert("ckanext.datapusher_plus.pii_regex_resource_id_or_alias", "");
app_main_section.insert("ckanext.datapusher_plus.pii_show_candidates", "false");
app_main_section.insert("ckanext.datapusher_plus.pii_quick_screen", "false");
app_main_section.insert("ckanext.datapusher_plus.qsv_bin", "/usr/local/bin/qsvdp");
app_main_section.insert("ckanext.datapusher_plus.preview_rows", "100");
app_main_section.insert("ckanext.datapusher_plus.download_timeout", "300");
app_main_section.insert(
"ckanext.datapusher_plus.max_content_length",
"1256000000000",
);
app_main_section.insert("ckanext.datapusher_plus.chunk_size", "16384");
app_main_section.insert("ckanext.datapusher_plus.default_excel_sheet", "0");
app_main_section.insert("ckanext.datapusher_plus.sort_and_dupe_check", "true");
app_main_section.insert("ckanext.datapusher_plus.dedup", "false");
app_main_section.insert("ckanext.datapusher_plus.unsafe_prefix", "unsafe_");
app_main_section.insert("ckanext.datapusher_plus.reserved_colnames", "_id");
app_main_section.insert("ckanext.datapusher_plus.prefer_dmy", "false");
app_main_section.insert("ckanext.datapusher_plus.ignore_file_hash", "true");
app_main_section.insert("ckanext.datapusher_plus.auto_index_threshold", "3");
app_main_section.insert("ckanext.datapusher_plus.auto_index_dates", "true");
app_main_section.insert("ckanext.datapusher_plus.auto_unique_index", "true");
app_main_section.insert("ckanext.datapusher_plus.summary_stats_options", "");
app_main_section.insert(
"ckanext.datapusher_plus.add_summary_stats_resource",
"false",
);
app_main_section.insert(
"ckanext.datapusher_plus.summary_stats_with_preview",
"false",
);
app_main_section.insert(
"ckanext.datapusher_plus.qsv_stats_string_max_length",
"32767",
);
app_main_section.insert(
"ckanext.datapusher_plus.qsv_dates_whitelist",
"date,time,due,open,close,created",
);
app_main_section.insert("ckanext.datapusher_plus.qsv_freq_limit", "10");
app_main_section.insert("ckanext.datapusher_plus.auto_alias", "true");
app_main_section.insert("ckanext.datapusher_plus.auto_alias_unique", "false");
app_main_section.insert("ckanext.datapusher_plus.copy_readbuffer_size", "1048576");
app_main_section.insert("ckanext.datapusher_plus.type_mapping", r#"{"String": "text", "Integer": "numeric","Float": "numeric","DateTime": "timestamp","Date": "date","NULL": "text"}"#);
app_main_section.insert("ckanext.datapusher_plus.auto_spatial_simplication", "true");
app_main_section.insert(
"ckanext.datapusher_plus.spatial_simplication_relative_tolerance",
"0.1",
);
app_main_section.insert("ckanext.datapusher_plus.latitude_fields", "latitude,lat");
app_main_section.insert(
"ckanext.datapusher_plus.longitude_fields",
"longitude,long,lon",
);
app_main_section.insert(
"ckanext.datapusher_plus.jinja2_bytecode_cache_dir",
"/tmp/jinja2_butecode_cache",
);
app_main_section.insert("ckanext.datapusher_plus.auto_unzip_one_file", "true");
app_main_section.insert(
"ckanext.datapusher_plus.api_token",
"<CKAN service account token for CKAN user with sysadmin privileges>",
);
app_main_section.insert(
"ckanext.datapusher_plus.describeGPT_api_key",
"<Token for OpenAI API compatible service>",
);
app_main_section.insert("ckanext.datapusher_plus.file_bin", "/usr/bin/file");
app_main_section.insert("ckanext.datapusher_plus.enable_druf", "false");
app_main_section.insert("ckanext.datapusher_plus.enable_form_redirect", "true");
conf.write_to_file("/etc/ckan/default/ckan.ini")?;
let resource_formats_str = std::fs::read_to_string(
"/usr/lib/ckan/default/src/ckan/ckan/config/resource_formats.json",
)?;
let mut resource_formats_val: serde_json::Value = serde_json::from_str(&resource_formats_str)?;
let all_resource_formats = resource_formats_val
.get_mut(0)
.unwrap()
.as_array_mut()
.unwrap();
all_resource_formats.push(json!([
"TAB",
"Tab Separated Values File",
"text/tab-separated-values",
[]
]));
std::fs::write(
"/usr/lib/ckan/default/src/ckan/ckan/config/resource_formats.json",
serde_json::to_string(&resource_formats_val)?,
)?;
cmd!(sh, "sudo locale-gen en_US.UTF-8").run()?;
cmd!(sh, "sudo update-locale").run()?;
let token_command_output = cmd!(
sh,
"ckan -c /etc/ckan/default/ckan.ini user token add {sysadmin_username} dpplus"
)
.read()?;
let tail_output = cmd!(sh, "tail -n 1").stdin(token_command_output).read()?;
let dpp_api_token = cmd!(sh, "tr -d '\t'").stdin(tail_output).read()?;
cmd!(sh, "ckan config-tool /etc/ckan/default/ckan.ini ckanext.datapusher_plus.api_token={dpp_api_token}").env("LC_ALL", "en_US.UTF-8").run()?;
cmd!(
sh,
"ckan -c /etc/ckan/default/ckan.ini db upgrade -p datapusher_plus"
)
.run()?;
println!(
"{}",
success_text(format!("{step_prefix} Installed DataPusher+ extension.").as_str())
);
Ok(())
}