Compare commits

...

16 commits

Author SHA1 Message Date
rzmk
a24d49042b docs: link to CKAN in README 2025-10-14 08:57:06 -04:00
rzmk
6af03d8c0b fix: resource_formats.json file path and use rust-ini instead of ckan config-tool 2025-10-14 08:49:14 -04:00
rzmk
cb0802a4b2 fix: refactor extension installs into separate steps 2025-10-14 07:55:44 -04:00
rzmk
1f2b68f4d2 fix(docs): icon-xs issue and quick-start href 2025-10-14 06:39:59 -04:00
rzmk
451184a914 feat(docs): add /docs/changelog and changelog for v0.3.0 2025-10-14 06:34:25 -04:00
Mueez Khan
b92058e451
docs: add site preview image to README 2025-10-14 06:14:55 -04:00
rzmk
9a74c987ee docs: add user-friendly README changes for new developers (#10, #11) 2025-10-14 06:13:55 -04:00
rzmk
65aff22b99 feat(docs): add options to quick-start, update home page, disable what-is for now 2025-10-14 05:47:11 -04:00
rzmk
1c0f3c2e15 feat: update to 0.3.0, remove --default, add --skip-interactive 2025-10-14 05:46:40 -04:00
rzmk
4808647cea chore: clarify value delimiter for extensions and features options 2025-10-14 05:15:10 -04:00
rzmk
0fab6f9f41 chore: remove custom git repo doc for now 2025-10-14 05:14:40 -04:00
rzmk
efdc0f93fd feat(docs): add uninstall subcommand to uninstall page, remove preset option 2025-10-14 05:13:48 -04:00
rzmk
e3da0a573d feat: add/fix uninstall subcommand, update info 2025-10-14 05:13:15 -04:00
rzmk
c16cdfa62c feat: add --uninstall flag 2025-10-14 04:42:01 -04:00
rzmk
56ae938e6c feat(docs): add interactivity to Builder, sync with command, add sonner toast 2025-10-14 04:23:31 -04:00
rzmk
25bb877fb6 feat: add CLI config options and refactor steps 2025-10-14 04:23:02 -04:00
24 changed files with 1071 additions and 469 deletions

2
Cargo.lock generated
View file

@ -126,7 +126,7 @@ checksum = "9555578bc9e57714c812a1f84e4fc5b4d21fcb063490c624de019f7464c91268"
[[package]]
name = "ckan-devstaller"
version = "0.1.0"
version = "0.3.0"
dependencies = [
"anyhow",
"clap",

View file

@ -1,6 +1,6 @@
[package]
name = "ckan-devstaller"
version = "0.1.0"
version = "0.3.0"
edition = "2024"
[dependencies]

View file

@ -1,42 +1,36 @@
# ckan-devstaller
`ckan-devstaller` attempts to install CKAN 2.11.3 from source using [ckan-compose](https://github.com/tino097/ckan-compose), intended for development use in a new Ubuntu 22.04 instance. The following are also installed and enabled by default:
<a href="https://ckan-devstaller.dathere.com"><img width="1165" height="668" alt="{1329F0BA-A29F-4BF8-BB6B-E3BA84FDAFCC}" src="https://github.com/user-attachments/assets/8f0cc4ef-d90d-4715-ba21-9083dff0c3ff" /></a>
- [DataStore extension](https://docs.ckan.org/en/2.11/maintaining/datastore.html)
- [ckanext-scheming extension](https://github.com/ckan/ckanext-scheming)
- [DataPusher+ extension](https://github.com/dathere/datapusher-plus)
[DRUF mode](https://github.com/dathere/datapusher-plus?tab=readme-ov-file#druf-dataset-resource-upload-first-workflow) is available but disabled by default. The [`datatablesview-plus` extension](https://github.com/dathere/ckanext-datatables-plus) is planned to be included in a future release.
`ckan-devstaller` attempts to install a [CKAN](https://ckan.org) instance using [ckan-compose](https://github.com/tino097/ckan-compose) for development usage in a new Ubuntu 22.04 instance.
## Quick start
You may find `ckan-devstaller` useful for:
> [!CAUTION]
> Make sure `ckan-devstaller` is run in a **new** Ubuntu 22.04 instance. Do NOT run `ckan-devstaller` in an existing instance that is important for your usage.
- Exploring CKAN for the first time without spending hours on installation steps
- Developing/Testing CKAN extensions and fixing bugs
- Trying a new CKAN version to test an upgrade from a legacy version
> [!WARNING]
> If you are using Ubuntu 22.04 on VirtualBox, you may need to add your user to the sudoers file before running the ckan-devstaller install script. Open a terminal in your virtual machine (VM), run `su -` and log in as the root user with the password you used to set up the VM, then type `sudo adduser <username> sudo` where `<username>` is your username then restart your VM and run the ckan-devstaller installer script.
`ckan-devstaller` was made to help speed up the installation time for CKAN and various extensions/features to boost development productivity.
> [!NOTE]
> The `/etc/ckan/default/ckan.ini` config file will have its comments removed for now. There are plans to fix this in a future release of `ckan-devstaller`.
**Get started at [ckan-devstaller.dathere.com](https://ckan-devstaller.dathere.com).**
> [!NOTE]
> Currently `ckan-devstaller` supports x86 architecture. ARM support is planned.
## Learn more about developing with CKAN
You have two common options to choose from for installation. Paste one of the following scripts into your new Ubuntu 22.04 instance's terminal.
You may find the following guides useful while developing with CKAN:
### Install with non-interactive mode (default config)
- [CKAN Hardware Requirements](https://github.com/ckan/ckan/wiki/Hardware-Requirements) - Learn what you need before installing CKAN
- [CKAN Sysadmin guide](https://docs.ckan.org/en/latest/sysadmin-guide.html) - Useful for CKAN instance administrators/sysadmins
- [CKAN Theming guide](https://docs.ckan.org/en/latest/theming/index.html) - Explore how to set up custom themes for your CKAN instance
- [CKAN Extending guide](https://docs.ckan.org/en/latest/extensions/index.html) - Develop CKAN extensions that can enhance your CKAN instance's functionality and add custom features
```bash
wget -O - https://github.com/dathere/ckan-devstaller/releases/download/0.2.1/install.bash | bash -s default
```
## What next?
### Install with interactive mode
- [Customize your config file](https://docs.ckan.org/en/latest/extensions/index.html)
- [Create test data](https://docs.ckan.org/en/latest/maintaining/getting-started.html#creating-test-data)
- [Visit ckan.org](https://ckan.org)
```bash
wget -O - https://github.com/dathere/ckan-devstaller/releases/download/0.2.1/install.bash | bash
```
## Demos
## `ckan-devstaller` demos
### Interactive customizable installation

View file

@ -38,33 +38,6 @@ export default function HomePage() {
<div className="relative mb-4">
<Hero />
</div>
<Cards>
<Card
icon={<ZapIcon />}
href="/docs/quick-start"
title="Quick start"
>
Get started with ckan-devstaller and install CKAN within minutes
</Card>
<Card icon={<BlocksIcon />} href="/docs/builder" title="Builder">
Customize your installation with an interactive web GUI
</Card>
<Card
icon={<HomeIcon />}
href="/docs/reference/installation-architecture"
title="Installation architecture"
>
Learn about where files are installed after running
ckan-devstaller
</Card>
<Card
icon={<GitMergeIcon />}
href="https://github.com/dathere/ckan-devstaller"
title="Source code"
>
View the source code of ckan-devstaller on GitHub
</Card>
</Cards>
</div>
</main>
</>
@ -72,6 +45,7 @@ export default function HomePage() {
}
function Hero() {
const { Card, Cards } = defaultMdxComponents;
return (
<div className="relative z-2 flex flex-col border-x border-t bg-fd-background/80 px-4 pt-12 max-md:text-center md:px-12 md:pt-16 [.uwu_&]:hidden overflow-hidden">
<div
@ -112,7 +86,7 @@ function Hero() {
</Link>
.
</p>
<div className="inline-flex items-center gap-3 max-md:mx-auto">
<div className="inline-flex items-center gap-3 max-md:mx-auto mb-8">
<Link
href="/docs"
className={cn(
@ -134,6 +108,33 @@ function Hero() {
Source Code
</Link>
</div>
<Cards>
<Card
icon={<ZapIcon />}
href="/docs"
title="Quick start"
>
Get started with ckan-devstaller and install CKAN within minutes
</Card>
<Card icon={<BlocksIcon />} href="/docs/builder" title="Builder">
Customize your installation with an interactive web GUI
</Card>
<Card
icon={<HomeIcon />}
href="/docs/reference/installation-architecture"
title="Installation architecture"
>
Learn about where files are installed after running
ckan-devstaller
</Card>
<Card
icon={<GitMergeIcon />}
href="https://github.com/dathere/ckan-devstaller"
title="Source code"
>
View the source code of ckan-devstaller on GitHub
</Card>
</Cards>
<PreviewImages />
</div>
);
@ -149,7 +150,7 @@ function PreviewImages() {
];
return (
<div className="mt-12 p-8 min-w-[800px] overflow-hidden xl:-mx-12 dark:[mask-image:linear-gradient(to_top,transparent,white_40px)]">
<div className="p-8 min-w-[800px] overflow-hidden xl:-mx-12 dark:[mask-image:linear-gradient(to_top,transparent,white_40px)]">
<div className="absolute flex flex-row left-1/2 -translate-1/2 bottom-4 z-2 p-1 rounded-full bg-fd-card border shadow-xl dark:shadow-fd-background">
{/* <div
role="none"

View file

@ -1,4 +1,5 @@
@import 'tailwindcss';
@import 'fumadocs-ui/css/neutral.css';
@import 'fumadocs-ui/css/ocean.css';
@import 'fumadocs-ui/css/preset.css';
@import "tailwindcss";
@import "fumadocs-ui/css/neutral.css";
@import "fumadocs-ui/css/ocean.css";
@import "fumadocs-ui/css/preset.css";
@import "tw-animate-css";

View file

@ -2,6 +2,7 @@ import "@/app/global.css";
import { RootProvider } from "fumadocs-ui/provider";
import localFont from "next/font/local";
import Script from "next/script";
import { Toaster } from "@/components/ui/sonner";
const inter = localFont({ src: "../lib/inter.ttf" });
@ -18,6 +19,7 @@ export default function Layout({ children }: LayoutProps<"/">) {
data-web-vitals="true"
strategy="afterInteractive"
/>
<Toaster closeButton richColors />
</body>
</html>
);

View file

@ -8,17 +8,20 @@
"@radix-ui/react-collapsible": "^1.1.12",
"@radix-ui/react-tabs": "^1.1.13",
"class-variance-authority": "^0.7.1",
"clsx": "^2.1.1",
"fumadocs-core": "15.8.1",
"fumadocs-mdx": "12.0.1",
"fumadocs-ui": "15.8.1",
"lucide-react": "^0.544.0",
"lucide-react": "^0.545.0",
"next": "15.5.4",
"next-themes": "^0.4.6",
"react": "^19.1.1",
"react-dom": "^19.1.1",
"sonner": "^2.0.7",
"tailwind-merge": "^3.3.1",
},
"devDependencies": {
"@biomejs/biome": "2.2.5",
"@biomejs/biome": "2.2.6",
"@tailwindcss/postcss": "^4.1.13",
"@types/mdx": "^2.0.13",
"@types/node": "24.5.2",
@ -26,6 +29,7 @@
"@types/react-dom": "^19.1.9",
"postcss": "^8.5.6",
"tailwindcss": "^4.1.13",
"tw-animate-css": "^1.4.0",
"typescript": "^5.9.2",
},
},
@ -33,23 +37,23 @@
"packages": {
"@alloc/quick-lru": ["@alloc/quick-lru@5.2.0", "", {}, "sha512-UrcABB+4bUrFABwbluTIBErXwvbsU/V7TZWfmbgJfbkwiBuziS9gxdODUyuiecfdGQ85jglMW6juS3+z5TsKLw=="],
"@biomejs/biome": ["@biomejs/biome@2.2.5", "", { "optionalDependencies": { "@biomejs/cli-darwin-arm64": "2.2.5", "@biomejs/cli-darwin-x64": "2.2.5", "@biomejs/cli-linux-arm64": "2.2.5", "@biomejs/cli-linux-arm64-musl": "2.2.5", "@biomejs/cli-linux-x64": "2.2.5", "@biomejs/cli-linux-x64-musl": "2.2.5", "@biomejs/cli-win32-arm64": "2.2.5", "@biomejs/cli-win32-x64": "2.2.5" }, "bin": { "biome": "bin/biome" } }, "sha512-zcIi+163Rc3HtyHbEO7CjeHq8DjQRs40HsGbW6vx2WI0tg8mYQOPouhvHSyEnCBAorfYNnKdR64/IxO7xQ5faw=="],
"@biomejs/biome": ["@biomejs/biome@2.2.6", "", { "optionalDependencies": { "@biomejs/cli-darwin-arm64": "2.2.6", "@biomejs/cli-darwin-x64": "2.2.6", "@biomejs/cli-linux-arm64": "2.2.6", "@biomejs/cli-linux-arm64-musl": "2.2.6", "@biomejs/cli-linux-x64": "2.2.6", "@biomejs/cli-linux-x64-musl": "2.2.6", "@biomejs/cli-win32-arm64": "2.2.6", "@biomejs/cli-win32-x64": "2.2.6" }, "bin": { "biome": "bin/biome" } }, "sha512-yKTCNGhek0rL5OEW1jbLeZX8LHaM8yk7+3JRGv08my+gkpmtb5dDE+54r2ZjZx0ediFEn1pYBOJSmOdDP9xtFw=="],
"@biomejs/cli-darwin-arm64": ["@biomejs/cli-darwin-arm64@2.2.5", "", { "os": "darwin", "cpu": "arm64" }, "sha512-MYT+nZ38wEIWVcL5xLyOhYQQ7nlWD0b/4mgATW2c8dvq7R4OQjt/XGXFkXrmtWmQofaIM14L7V8qIz/M+bx5QQ=="],
"@biomejs/cli-darwin-arm64": ["@biomejs/cli-darwin-arm64@2.2.6", "", { "os": "darwin", "cpu": "arm64" }, "sha512-UZPmn3M45CjTYulgcrFJFZv7YmK3pTxTJDrFYlNElT2FNnkkX4fsxjExTSMeWKQYoZjvekpH5cvrYZZlWu3yfA=="],
"@biomejs/cli-darwin-x64": ["@biomejs/cli-darwin-x64@2.2.5", "", { "os": "darwin", "cpu": "x64" }, "sha512-FLIEl73fv0R7dI10EnEiZLw+IMz3mWLnF95ASDI0kbx6DDLJjWxE5JxxBfmG+udz1hIDd3fr5wsuP7nwuTRdAg=="],
"@biomejs/cli-darwin-x64": ["@biomejs/cli-darwin-x64@2.2.6", "", { "os": "darwin", "cpu": "x64" }, "sha512-HOUIquhHVgh/jvxyClpwlpl/oeMqntlteL89YqjuFDiZ091P0vhHccwz+8muu3nTyHWM5FQslt+4Jdcd67+xWQ=="],
"@biomejs/cli-linux-arm64": ["@biomejs/cli-linux-arm64@2.2.5", "", { "os": "linux", "cpu": "arm64" }, "sha512-5DjiiDfHqGgR2MS9D+AZ8kOfrzTGqLKywn8hoXpXXlJXIECGQ32t+gt/uiS2XyGBM2XQhR6ztUvbjZWeccFMoQ=="],
"@biomejs/cli-linux-arm64": ["@biomejs/cli-linux-arm64@2.2.6", "", { "os": "linux", "cpu": "arm64" }, "sha512-BpGtuMJGN+o8pQjvYsUKZ+4JEErxdSmcRD/JG3mXoWc6zrcA7OkuyGFN1mDggO0Q1n7qXxo/PcupHk8gzijt5g=="],
"@biomejs/cli-linux-arm64-musl": ["@biomejs/cli-linux-arm64-musl@2.2.5", "", { "os": "linux", "cpu": "arm64" }, "sha512-5Ov2wgAFwqDvQiESnu7b9ufD1faRa+40uwrohgBopeY84El2TnBDoMNXx6iuQdreoFGjwW8vH6k68G21EpNERw=="],
"@biomejs/cli-linux-arm64-musl": ["@biomejs/cli-linux-arm64-musl@2.2.6", "", { "os": "linux", "cpu": "arm64" }, "sha512-TjCenQq3N6g1C+5UT3jE1bIiJb5MWQvulpUngTIpFsL4StVAUXucWD0SL9MCW89Tm6awWfeXBbZBAhJwjyFbRQ=="],
"@biomejs/cli-linux-x64": ["@biomejs/cli-linux-x64@2.2.5", "", { "os": "linux", "cpu": "x64" }, "sha512-fq9meKm1AEXeAWan3uCg6XSP5ObA6F/Ovm89TwaMiy1DNIwdgxPkNwxlXJX8iM6oRbFysYeGnT0OG8diCWb9ew=="],
"@biomejs/cli-linux-x64": ["@biomejs/cli-linux-x64@2.2.6", "", { "os": "linux", "cpu": "x64" }, "sha512-1HaM/dpI/1Z68zp8ZdT6EiBq+/O/z97a2AiHMl+VAdv5/ELckFt9EvRb8hDHpk8hUMoz03gXkC7VPXOVtU7faA=="],
"@biomejs/cli-linux-x64-musl": ["@biomejs/cli-linux-x64-musl@2.2.5", "", { "os": "linux", "cpu": "x64" }, "sha512-AVqLCDb/6K7aPNIcxHaTQj01sl1m989CJIQFQEaiQkGr2EQwyOpaATJ473h+nXDUuAcREhccfRpe/tu+0wu0eQ=="],
"@biomejs/cli-linux-x64-musl": ["@biomejs/cli-linux-x64-musl@2.2.6", "", { "os": "linux", "cpu": "x64" }, "sha512-1ZcBux8zVM3JhWN2ZCPaYf0+ogxXG316uaoXJdgoPZcdK/rmRcRY7PqHdAos2ExzvjIdvhQp72UcveI98hgOog=="],
"@biomejs/cli-win32-arm64": ["@biomejs/cli-win32-arm64@2.2.5", "", { "os": "win32", "cpu": "arm64" }, "sha512-xaOIad4wBambwJa6mdp1FigYSIF9i7PCqRbvBqtIi9y29QtPVQ13sDGtUnsRoe6SjL10auMzQ6YAe+B3RpZXVg=="],
"@biomejs/cli-win32-arm64": ["@biomejs/cli-win32-arm64@2.2.6", "", { "os": "win32", "cpu": "arm64" }, "sha512-h3A88G8PGM1ryTeZyLlSdfC/gz3e95EJw9BZmA6Po412DRqwqPBa2Y9U+4ZSGUAXCsnSQE00jLV8Pyrh0d+jQw=="],
"@biomejs/cli-win32-x64": ["@biomejs/cli-win32-x64@2.2.5", "", { "os": "win32", "cpu": "x64" }, "sha512-F/jhuXCssPFAuciMhHKk00xnCAxJRS/pUzVfXYmOMUp//XW7mO6QeCjsjvnm8L4AO/dG2VOB0O+fJPiJ2uXtIw=="],
"@biomejs/cli-win32-x64": ["@biomejs/cli-win32-x64@2.2.6", "", { "os": "win32", "cpu": "x64" }, "sha512-yx0CqeOhPjYQ5ZXgPfu8QYkgBhVJyvWe36as7jRuPrKPO5ylVDfwVtPQ+K/mooNTADW0IhxOZm3aPu16dP8yNQ=="],
"@emnapi/runtime": ["@emnapi/runtime@1.5.0", "", { "dependencies": { "tslib": "^2.4.0" } }, "sha512-97/BJ3iXHww3djw6hYIfErCZFee7qCtrneuLa20UXFCOTCfBM2cvQHjWJ2EG0s0MtdNwInarqCTz35i4wWXHsQ=="],
@ -495,7 +499,7 @@
"lru-cache": ["lru-cache@11.2.2", "", {}, "sha512-F9ODfyqML2coTIsQpSkRHnLSZMtkU8Q+mSfcaIyKwy58u+8k5nvAYeiNhsyMARvzNcXJ9QfWVrcPsC9e9rAxtg=="],
"lucide-react": ["lucide-react@0.544.0", "", { "peerDependencies": { "react": "^16.5.1 || ^17.0.0 || ^18.0.0 || ^19.0.0" } }, "sha512-t5tS44bqd825zAW45UQxpG2CvcC4urOwn2TrwSH8u+MjeE+1NnWl6QqeQ/6NdjMqdOygyiT9p3Ev0p1NJykxjw=="],
"lucide-react": ["lucide-react@0.545.0", "", { "peerDependencies": { "react": "^16.5.1 || ^17.0.0 || ^18.0.0 || ^19.0.0" } }, "sha512-7r1/yUuflQDSt4f1bpn5ZAocyIxcTyVyBBChSVtBKn5M+392cPmI5YJMWOJKk/HUWGm5wg83chlAZtCcGbEZtw=="],
"magic-string": ["magic-string@0.30.19", "", { "dependencies": { "@jridgewell/sourcemap-codec": "^1.5.5" } }, "sha512-2N21sPY9Ws53PZvsEpVtNuSW+ScYbQdp4b9qUaL+9QkHUrGFKo56Lg9Emg5s9V/qrtNBmiR01sYhUOwu3H+VOw=="],
@ -689,6 +693,8 @@
"shiki": ["shiki@3.13.0", "", { "dependencies": { "@shikijs/core": "3.13.0", "@shikijs/engine-javascript": "3.13.0", "@shikijs/engine-oniguruma": "3.13.0", "@shikijs/langs": "3.13.0", "@shikijs/themes": "3.13.0", "@shikijs/types": "3.13.0", "@shikijs/vscode-textmate": "^10.0.2", "@types/hast": "^3.0.4" } }, "sha512-aZW4l8Og16CokuCLf8CF8kq+KK2yOygapU5m3+hoGw0Mdosc6fPitjM+ujYarppj5ZIKGyPDPP1vqmQhr+5/0g=="],
"sonner": ["sonner@2.0.7", "", { "peerDependencies": { "react": "^18.0.0 || ^19.0.0 || ^19.0.0-rc", "react-dom": "^18.0.0 || ^19.0.0 || ^19.0.0-rc" } }, "sha512-W6ZN4p58k8aDKA4XPcx2hpIQXBRAgyiWVkYhT7CvK6D3iAu7xjvVyhQHg2/iaKJZ1XVJ4r7XuwGL+WGEK37i9w=="],
"source-map": ["source-map@0.7.6", "", {}, "sha512-i5uvt8C3ikiWeNZSVZNWcfZPItFQOsYTUAOkcUPGd8DqDy1uOUikjt5dG+uRlwyvR108Fb9DOd4GvXfT0N2/uQ=="],
"source-map-js": ["source-map-js@1.2.1", "", {}, "sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA=="],
@ -721,6 +727,8 @@
"tslib": ["tslib@2.8.1", "", {}, "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w=="],
"tw-animate-css": ["tw-animate-css@1.4.0", "", {}, "sha512-7bziOlRqH0hJx80h/3mbicLW7o8qLsH5+RaLR2t+OHM3D0JlWGODQKQ4cxbK7WlvmUxpcj6Kgu6EKqjrGFe3QQ=="],
"typescript": ["typescript@5.9.2", "", { "bin": { "tsc": "bin/tsc", "tsserver": "bin/tsserver" } }, "sha512-CWBzXQrc/qOkhidw1OzBTQuYRbfyxDXJMVJ1XNwUHGROVmuaeiEm3OslpZ1RV96d7SKKjZKrSJu3+t/xlw3R9A=="],
"undici-types": ["undici-types@7.12.0", "", {}, "sha512-goOacqME2GYyOZZfb5Lgtu+1IDmAlAEu5xnD3+xTzS10hT0vzpf0SPjkXwAw9Jm+4n/mQGDP3LO8CPbYROeBfQ=="],

22
docs/components.json Normal file
View file

@ -0,0 +1,22 @@
{
"$schema": "https://ui.shadcn.com/schema.json",
"style": "default",
"rsc": true,
"tsx": true,
"tailwind": {
"config": "",
"css": "app/global.css",
"baseColor": "neutral",
"cssVariables": true,
"prefix": ""
},
"iconLibrary": "lucide",
"aliases": {
"components": "@/components",
"utils": "@/lib/utils",
"ui": "@/components/ui",
"lib": "@/lib",
"hooks": "@/hooks"
},
"registries": {}
}

View file

@ -0,0 +1,107 @@
import defaultMdxComponents from "fumadocs-ui/mdx";
import { SailboatIcon, TerminalSquareIcon } from "lucide-react";
import { Config, selectedCardClasses } from "../builder";
import { toast } from "sonner";
const getExtensionClassName = (config: Config, extensionName: string) => {
return config.extensions.includes(extensionName) ? selectedCardClasses : "";
};
const updateExtensions = (
config: Config,
setConfig: any,
extensions: string[] | string,
mode?: "add" | "remove",
) => {
const extensionsArray = Array.isArray(extensions) ? extensions : [extensions];
if (mode === "add") {
setConfig({
...config,
extensions: [...new Set([...config.extensions, ...extensionsArray])],
});
return;
}
for (const extensionName of extensionsArray) {
if (config.extensions.includes(extensionName))
setConfig({
...config,
extensions: config.extensions.filter(
(extension) => extension !== extensionName,
),
});
else if (!config.extensions.includes(extensionName))
setConfig({
...config,
extensions: [...config.extensions, extensionName],
});
}
};
export default function CKANExtensionsBuilderSection({
config,
setConfig,
}: {
config: Config;
setConfig: any;
}) {
const { Card, Cards } = defaultMdxComponents;
return (
<>
<h3>CKAN extensions</h3>
<Cards>
<Card
className={getExtensionClassName(config, "ckanext-scheming")}
icon={<TerminalSquareIcon />}
title="ckanext-scheming"
onClick={() => {
if (
config.extensions.includes("DataPusher+") &&
config.extensions.includes("ckanext-scheming")
) {
toast.error(
"You cannot remove the ckanext-scheming extension because the DataPusher+ extension depends on it.",
);
return;
}
updateExtensions(config, setConfig, "ckanext-scheming");
}}
></Card>
<Card
className={getExtensionClassName(config, "DataStore")}
icon={<TerminalSquareIcon />}
title="DataStore"
onClick={() => {
if (
config.extensions.includes("DataPusher+") &&
config.extensions.includes("DataStore")
) {
toast.error(
"You cannot remove the DataStore extension because the DataPusher+ extension depends on it.",
);
return;
}
updateExtensions(config, setConfig, "DataStore");
}}
></Card>
<Card
className={getExtensionClassName(config, "DataPusher+")}
icon={<TerminalSquareIcon />}
title="DataPusher+"
onClick={() => {
if (config.extensions.includes("DataPusher+")) {
updateExtensions(config, setConfig, "DataPusher+");
} else {
updateExtensions(
config,
setConfig,
["DataPusher+", "ckanext-scheming", "DataStore"],
"add",
);
}
}}
></Card>
</Cards>
</>
);
}

View file

@ -0,0 +1,39 @@
import defaultMdxComponents from "fumadocs-ui/mdx";
import { SailboatIcon } from "lucide-react";
import { selectedCardClasses } from "../builder";
export default function CKANVersionBuilderSection({ config, setConfig }: any) {
const { Card, Cards } = defaultMdxComponents;
return (
<>
<h3>CKAN version</h3>
<Cards>
<Card
icon={<SailboatIcon />}
title="2.11.3"
className={
config.ckanVersion === "2.11.3"
? selectedCardClasses
: "cursor-pointer"
}
onClick={() => {
setConfig({ ...config, ckanVersion: "2.11.3" });
}}
></Card>
<Card
icon={<SailboatIcon />}
title="2.10.8"
className={
config.ckanVersion === "2.10.8"
? selectedCardClasses
: "cursor-pointer"
}
onClick={() => {
setConfig({ ...config, ckanVersion: "2.10.8" });
}}
></Card>
</Cards>
</>
);
}

View file

@ -0,0 +1,48 @@
import defaultMdxComponents from "fumadocs-ui/mdx";
import { SailboatIcon, TerminalSquareIcon } from "lucide-react";
import { Config, selectedCardClasses } from "../builder";
const getFeatureClassName = (config: Config, featureName: string) => {
return config.features.includes(featureName) ? selectedCardClasses : "";
};
const updateFeatures = (
config: Config,
setConfig: any,
featureName: string,
) => {
if (config.features.includes(featureName))
setConfig({
...config,
features: config.features.filter((feature) => feature !== featureName),
});
else setConfig({ ...config, features: [...config.features, featureName] });
};
export default function FeaturesBuilderSection({
config,
setConfig,
}: {
config: Config;
setConfig: any;
}) {
const { Card, Cards } = defaultMdxComponents;
return (
<>
<h3>Features</h3>
<Cards>
<Card
className={getFeatureClassName(config, "enable-ssh")}
icon={<TerminalSquareIcon />}
title="Enable SSH"
onClick={() => {
updateFeatures(config, setConfig, "enable-ssh");
}}
>
Installs the openssh-server package.
</Card>
</Cards>
</>
);
}

View file

@ -0,0 +1,63 @@
import { Config, selectedCardClasses } from "../builder";
import { BarChartBigIcon, SailboatIcon } from "lucide-react";
import defaultMdxComponents from "fumadocs-ui/mdx";
export default function PresetsBuilderSection({
config,
setConfig,
}: {
config: Config;
setConfig: any;
}) {
const { Card, Cards } = defaultMdxComponents;
return (
<>
<h3>Presets</h3>
<Cards className="grid-cols-2">
<Card
className={
config.preset === "ckan-only" &&
config.extensions.length === 0 &&
config.features.length === 0
? selectedCardClasses
: "cursor-pointer"
}
icon={<SailboatIcon />}
title="CKAN-only"
onClick={() => {
setConfig({
...config,
preset: "ckan-only",
extensions: [],
features: [],
});
}}
>
Installs CKAN with ckan-compose. No CKAN extensions and extra features
are installed.
</Card>
<Card
className={
config.preset === "dathere-default"
? selectedCardClasses
: "cursor-pointer"
}
icon={<BarChartBigIcon />}
title="datHere Default"
onClick={() => {
setConfig({
...config,
preset: "dathere-default",
ckanVersion: "2.11.3",
extensions: ["ckanext-scheming", "DataStore", "DataPusher+"],
features: ["enable-ssh"],
});
}}
>
datHere's default preset featuring the DataPusher+ extension.
</Card>
</Cards>
</>
);
}

View file

@ -8,23 +8,49 @@ import {
SailboatIcon,
TerminalSquareIcon,
} from "lucide-react";
import { useState } from "react";
import { useEffect, useState } from "react";
import PresetsBuilderSection from "./builder-sections/presets";
import CKANVersionBuilderSection from "./builder-sections/ckan-version";
import CKANExtensionsBuilderSection from "./builder-sections/ckan-extensions";
import FeaturesBuilderSection from "./builder-sections/features";
type Config = {
export type Config = {
preset: string | undefined;
ckanVersion: string;
extensions: string[];
features: string[];
};
export const selectedCardClasses =
"bg-blue-100 dark:bg-blue-950 border-blue-300 dark:border-blue-900 border-2";
export default function Builder() {
const { Card, Cards } = defaultMdxComponents;
const [command, setCommand] = useState("./ckan-devstaller");
const [config, setConfig] = useState<Config>({
preset: undefined,
preset: "ckan-only",
ckanVersion: "2.11.3",
extensions: [],
features: [],
});
// Update command string when user changes configuration
useEffect(() => {
const ckanVersionString = `--ckan-version ${config.ckanVersion}`;
const extensionsString =
config.extensions.length > 0
? ` \\\n--extensions ${config.extensions.join(" ")}`
: undefined;
const featuresString =
config.features.length > 0
? ` \\\n--features ${config.features.join(" ")}`
: undefined;
setCommand(
`./ckan-devstaller \\
${ckanVersionString}${extensionsString ? extensionsString : ""}${featuresString ? featuresString : ""}`,
);
}, [config]);
return (
<div className="grid grid-cols-3 gap-4">
<div className="col-span-1 border-r-2 pr-4">
@ -35,72 +61,38 @@ export default function Builder() {
</CodeBlock>
<h2>Selected configuration</h2>
<div>
<strong>CKAN version</strong>: 2.11.3
<strong>CKAN version</strong>: {config.ckanVersion}
<br />
<br />
<strong>Extensions:</strong>
<ul>
<li>DataStore</li>
<li>ckanext-scheming</li>
<li>DataPusher+</li>
</ul>
<strong>Extra features:</strong>
<ul>
<li>Enable SSH</li>
</ul>
{config.extensions.length > 0 && (
<>
<strong>Extensions:</strong>
<ul>
{config.extensions.map((extension) => (
<li key={extension}>{extension}</li>
))}
</ul>
</>
)}
{config.features.length > 0 && (
<>
<strong>Features:</strong>
<ul>
{config.features.map((feature) => (
<li key={feature}>{feature}</li>
))}
</ul>
</>
)}
</div>
</div>
</div>
<div className="col-span-2">
<h2>Configuration options</h2>
<h3>Presets</h3>
<Cards className="grid-cols-2">
<Card
className="bg-blue-100 dark:bg-blue-950 border-blue-300 dark:border-blue-900 border-2"
icon={<SailboatIcon />}
title="CKAN-only"
>
Installs CKAN with ckan-compose.
</Card>
<Card icon={<BlocksIcon />} title="CKAN and the DataStore extension">
Installs CKAN and the DataStore extension.
</Card>
<Card icon={<BarChartBigIcon />} title="datHere Default">
Installs CKAN, the DataStore extension, the ckanext-scheming
extension, and the DataPusher+ extension.
</Card>
</Cards>
<h3>CKAN version</h3>
<Cards>
<Card icon={<SailboatIcon />} title="2.11.3"></Card>
<Card icon={<SailboatIcon />} title="2.10.8"></Card>
<Card
icon={<SailboatIcon />}
title="Install a different version"
></Card>
<Card
icon={<SailboatIcon />}
title="Clone from remote Git repository"
></Card>
</Cards>
<h3>CKAN extensions</h3>
<Cards>
<Card icon={<TerminalSquareIcon />} title="ckanext-scheming"></Card>
<Card icon={<TerminalSquareIcon />} title="ckanext-gztr"></Card>
<Card icon={<TerminalSquareIcon />} title="DataStore"></Card>
<Card icon={<TerminalSquareIcon />} title="DataPusher+"></Card>
<Card icon={<TerminalSquareIcon />} title="ckanext-spatial"></Card>
<Card icon={<TerminalSquareIcon />} title="Custom extension"></Card>
</Cards>
<h3>Extra features</h3>
<Cards>
<Card icon={<TerminalSquareIcon />} title="Enable SSH">
Installs openssh-server and net-tools.
</Card>
<Card icon={<TerminalSquareIcon />} title="Run a Bash script">
Run a Bash script before or after any step during the installation.
</Card>
</Cards>
<PresetsBuilderSection config={config} setConfig={setConfig} />
<CKANVersionBuilderSection config={config} setConfig={setConfig} />
<CKANExtensionsBuilderSection config={config} setConfig={setConfig} />
<FeaturesBuilderSection config={config} setConfig={setConfig} />
</div>
</div>
);

View file

@ -20,6 +20,7 @@ const buttonVariants = cva(
sm: "h-9 px-3",
lg: "h-11 px-6",
xs: "px-2 py-1.5 text-xs",
"icon-xs": "p-1 [&_svg]:size-4"
},
},
defaultVariants: {

View file

@ -0,0 +1,40 @@
"use client";
import {
CircleCheckIcon,
InfoIcon,
Loader2Icon,
OctagonXIcon,
TriangleAlertIcon,
} from "lucide-react";
import { useTheme } from "next-themes";
import { Toaster as Sonner, ToasterProps } from "sonner";
const Toaster = ({ ...props }: ToasterProps) => {
const { theme = "system" } = useTheme();
return (
<Sonner
theme={theme as ToasterProps["theme"]}
className="toaster group"
icons={{
success: <CircleCheckIcon className="size-4" />,
info: <InfoIcon className="size-4" />,
warning: <TriangleAlertIcon className="size-4" />,
error: <OctagonXIcon className="size-4" />,
loading: <Loader2Icon className="size-4 animate-spin" />,
}}
style={
{
"--normal-bg": "var(--popover)",
"--normal-text": "var(--popover-foreground)",
"--normal-border": "var(--border)",
"--border-radius": "var(--radius)",
} as React.CSSProperties
}
{...props}
/>
);
};
export { Toaster };

View file

@ -0,0 +1,37 @@
---
title: Changelog for ckan-devstaller v0.3.0 (2025-10-14)
---
Since v0.2.1 of ckan-devstaller, there have been many new features and changes now available in v0.3.0.
## New web app: ckan-devstaller.dathere.com
We've released a new web app [ckan-devstaller.dathere.com](https://ckan-devstaller.dathere.com) as the primary documentation site for ckan-devstaller.
## Builder page
There is now an interactive web GUI, the [Builder](/docs/builder), for users to customize their CKAN installation before copying the (now updated) ckan-devstaller command and running it on their terminal. This helps resolve issue [#6](https://github.com/dathere/ckan-devstaller/issues/11).
## Updated Quick Start page
The [Quick Start](/docs) page now includes three options for suggested installation methods:
1. Use the interactive [Builder](/docs/builder).
2. Install the "CKAN-only" preset with a script which installs the latest stable version of CKAN and ckan-compose (with optional non-interactive script).
3. Install the "datHere default" preset with a script which installs the latest stable version of CKAN and ckan-compose (with optional non-interactive script) along with the DataStore, ckanext-scheming, and DataPusher+ extensions and also installs the `openssh-server` package.
## Installation architecture page
There is now an [Installation Architecture](/docs/reference/installation-architecture) page in the Reference section of the web app that provides a visual representaion of where `ckan-devstaller` installs relevant files/folders.
## Uninstall CKAN page
There is now an [Uninstall CKAN](/docs/tutorials/uninstall-ckan) page in the Tutorials section of the web app that helps users understand how to uninstall their newly installed CKAN installation. This includes the option to either use the new `ckan-devstaller uninstall` subcommand or run the script directly.
## README update
The README on the [ckan-devstaller GitHub repository](https://github.com/dathere/ckan-devstaller) has been updated to have a more user-friendly focus for users and developers that may be new to CKAN thanks to the suggestions by [@drw](https://github.com/drw) in issues [#10](https://github.com/dathere/ckan-devstaller/issues/10) and [#11](https://github.com/dathere/ckan-devstaller/issues/11).
## Changelog section
We've added a Changelog section to the web app to denote new changes to `ckan-devstaller` for each release.

View file

@ -15,28 +15,56 @@ import { Accordion, Accordions } from 'fumadocs-ui/components/accordion';
<Accordion title="Note for ARM64 users">Currently `ckan-devstaller` supports `x86_64` architecture. `ARM64` support is planned.</Accordion>
</Accordions>
Currently you have two options to choose from for installation. Paste one of the following scripts into your new Ubuntu 22.04 instance's terminal:
---
## (Option 1/2) Install with interactive mode
You have several options to choose from for installation. Here are a few:
## [1/3] Customize your CKAN installation with the Builder (Recommended)
<Card
icon={<BlocksIcon />}
href="/docs/builder"
title="Builder"
>
Click here to customize your CKAN installation with an interactive web GUI
</Card>
## [2/3] Install the "CKAN-only" preset
By running the following script, ckan-devstaller will be downloaded and the default configuration for installing CKAN 2.11.3 with ckan-compose will be selected. You can then customize your configuration interactively in your terminal after running this script.
```bash
wget -O - https://github.com/dathere/ckan-devstaller/releases/download/0.2.1/install.bash | bash -s default
wget -O - https://github.com/dathere/ckan-devstaller/releases/download/0.3.0/install.bash | bash
```
## (Option 2/2) Install with non-interactive mode with a specific config
If you'd rather skip the interactivity and go straight to installation, then run the following script instead:
The following script will install the following:
```bash
wget -O - https://github.com/dathere/ckan-devstaller/releases/download/0.3.0/install.bash | bash -s skip-interactive
```
## [3/3] Install the "datHere Default" preset
The following script will download ckan-devstaller and select the following configuration:
- CKAN 2.11.3
- [ckan-compose](https://github.com/tino097/ckan-compose/tree/ckan-devstaller)
- [DataStore extension](https://docs.ckan.org/en/2.11/maintaining/datastore.html)
- [ckanext-scheming extension](https://github.com/ckan/ckanext-scheming)
- [DataPusher+ extension](https://github.com/dathere/datapusher-plus)
- Install the `openssh-server` package for allowing SSH capability
- [DRUF mode](https://github.com/dathere/datapusher-plus?tab=readme-ov-file#druf-dataset-resource-upload-first-workflow) for DataPusher+ is available but disabled by default.
[DRUF mode](https://github.com/dathere/datapusher-plus?tab=readme-ov-file#druf-dataset-resource-upload-first-workflow) for DataPusher+ is available but disabled by default.
You can then customize your configuration interactively in your terminal after running this script.
```bash
wget -O - https://github.com/dathere/ckan-devstaller/releases/download/0.2.1/install.bash | bash -s default
wget -O - https://github.com/dathere/ckan-devstaller/releases/download/0.3.0/install.bash | bash -s dathere-default
```
If you'd rather skip the interactivity and go straight to installation, then run the following script instead:
```bash
wget -O - https://github.com/dathere/ckan-devstaller/releases/download/0.3.0/install.bash | bash -s dathere-default skip-interactive
```
## Learn more

View file

@ -1,3 +1,11 @@
{
"pages": ["---Introduction---", "index", "what-is-ckan-devstaller", "builder", "---Further reading---", "tutorials", "reference"]
"pages": [
"---Introduction---",
"index",
"builder",
"---Further reading---",
"tutorials",
"reference",
"changelog"
]
}

View file

@ -5,7 +5,13 @@ description: How to uninstall CKAN after having installed with ckan-devstaller
You may want to uninstall CKAN and related files after having ran ckan-devstaller. This can be useful if you want to re-run ckan-devstaller with a different configuration or are developing ckan-devstaller.
Run the following script to uninstall CKAN and files related to ckan-devstaller:
The uninstallation process can be done by running:
```bash
./ckan-devstaller uninstall
```
The following script will be ran to uninstall CKAN and files related to ckan-devstaller:
```bash
sudo rm -rf /usr/lib/ckan

6
docs/lib/utils.ts Normal file
View file

@ -0,0 +1,6 @@
import { clsx, type ClassValue } from "clsx"
import { twMerge } from "tailwind-merge"
export function cn(...inputs: ClassValue[]) {
return twMerge(clsx(inputs))
}

View file

@ -15,17 +15,20 @@
"@radix-ui/react-collapsible": "^1.1.12",
"@radix-ui/react-tabs": "^1.1.13",
"class-variance-authority": "^0.7.1",
"clsx": "^2.1.1",
"fumadocs-core": "15.8.1",
"fumadocs-mdx": "12.0.1",
"fumadocs-ui": "15.8.1",
"lucide-react": "^0.544.0",
"lucide-react": "^0.545.0",
"next": "15.5.4",
"next-themes": "^0.4.6",
"react": "^19.1.1",
"react-dom": "^19.1.1",
"sonner": "^2.0.7",
"tailwind-merge": "^3.3.1"
},
"devDependencies": {
"@biomejs/biome": "2.2.5",
"@biomejs/biome": "2.2.6",
"@tailwindcss/postcss": "^4.1.13",
"@types/mdx": "^2.0.13",
"@types/node": "24.5.2",
@ -33,6 +36,7 @@
"@types/react-dom": "^19.1.9",
"postcss": "^8.5.6",
"tailwindcss": "^4.1.13",
"tw-animate-css": "^1.4.0",
"typescript": "^5.9.2"
}
}

View file

@ -11,18 +11,25 @@ sudo apt install curl -y
cd ~/
# Download the ckan-devstaller binary file
curl -LO https://github.com/dathere/ckan-devstaller/releases/download/0.2.1/ckan-devstaller
curl -LO https://github.com/dathere/ckan-devstaller/releases/download/0.3.0/ckan-devstaller
# Add execute permission to ckan-devstaller binary file
sudo chmod +x ./ckan-devstaller
# Run the ckan-devstaller binary file
# If the user provides an argument "default", run ckan-devstaller in non-interactive mode with the default config
# Otherwise run ckan-devstaller in interactive mode
flag=$1
# Run the ckan-devstaller binary file with the specified preset and (non-)interactive mode
preset=$1
skip_interactive=$2
if [ $flag == "default" ]; then
./ckan-devstaller --default
if [ $preset == "dathere-default" ]; then
if [ $skip_interactive == "skip-interactive" ]; then
./ckan-devstaller --ckan-version 2.11.3 --extensions ckanext-scheming DataStore DataPusher+ --features enable-ssh --skip-interactive
else
./ckan-devstaller --ckan-version 2.11.3 --extensions ckanext-scheming DataStore DataPusher+ --features enable-ssh
fi
else
./ckan-devstaller
if [ $preset == "skip-interactive" ]; then
./ckan-devstaller --skip-interactive
else
./ckan-devstaller
fi
fi

View file

@ -4,26 +4,49 @@ mod styles;
use crate::{
questions::{question_ckan_version, question_ssh, question_sysadmin},
styles::{highlighted_text, important_text, step_text, success_text},
steps::{
step_install_ahoy, step_install_and_run_ckan_compose,
step_install_ckanext_scheming_extension, step_install_curl,
step_install_datapusher_plus_extension, step_install_datastore_extension,
step_install_docker, step_install_openssh, step_package_updates,
},
styles::{important_text, step_text, success_text},
};
use anyhow::Result;
use clap::Parser;
use clap::{Parser, Subcommand};
use human_panic::{metadata, setup_panic};
use inquire::Confirm;
use serde_json::json;
use std::{path::PathBuf, str::FromStr};
use xshell::cmd;
use xshell_venv::{Shell, VirtualEnv};
/// ckan-devstaller CLI
#[derive(Parser, Debug)]
/// CLI to help install a CKAN instance for development within minutes. Learn more at: https://ckan-devstaller.dathere.com
#[derive(Parser)]
#[command(version, about, long_about = None)]
struct Args {
/// Skip interactive steps and install CKAN with default features
/// Skip interactive steps
#[arg(short, long)]
default: bool,
skip_interactive: bool,
#[arg(short, long)]
/// CKAN version to install defined by semantic versioning from official releases from https://github.com/ckan/ckan
ckan_version: Option<String>,
/// List of CKAN extensions to install, separated by spaces
#[arg(short, long, value_parser, num_args = 1.., value_delimiter = ' ')]
extensions: Option<Vec<String>>,
/// List of custom features, separated by spaces
#[arg(short, long, value_parser, num_args = 1.., value_delimiter = ' ')]
features: Option<Vec<String>>,
#[command(subcommand)]
command: Option<Commands>,
}
#[derive(Subcommand)]
enum Commands {
/// Attempt to uninstall CKAN and related ckan-devstaller installation files
Uninstall {},
}
#[derive(Clone)]
struct Sysadmin {
username: String,
password: String,
@ -45,37 +68,91 @@ fn main() -> Result<()> {
.homepage("https://dathere.com")
.support("- Create a support ticket at https://support.dathere.com or report an issue at https://github.com/dathere/ckan-devstaller"));
let args = Args::parse();
// Set up default config
let args = Args::parse();
let sh = Shell::new()?;
let username = cmd!(sh, "whoami").read()?;
steps::step_intro();
let default_config_text = r#"
The default configuration for ckan-devstaller does the following:
- Install openssh-server to enable SSH access
- Install ckan-compose (https://github.com/tino097/ckan-compose) which sets up the CKAN backend (PostgreSQL, SOLR, Redis)
- Install CKAN v2.11.3
- Install the DataStore extension
- Install the ckanext-scheming extension
- Install the DataPusher+ extension
- Disable DRUF mode for DataPusher+
"#;
println!("{default_config_text}");
let answer_customize = if args.default {
false
} else {
Confirm::new(
"Would you like to customize any of these features for your CKAN installation?",
if matches!(&args.command, Some(Commands::Uninstall {})) {
let uninstall_confirmation = Confirm::new(
"Are you sure you want to uninstall CKAN and related files from ckan-devstaller?",
)
.prompt()?
};
.with_help_message(
r#"The following commands are ran when attempting the uninstall:
sudo rm -rf /usr/lib/ckan
sudo rm -rf /etc/ckan
cd ~/
rm -rf qsv*
rm -rf README ckan-compose ahoy dpp_default_config.ini get-docker.sh permissions.sql"#,
)
.prompt()?;
if uninstall_confirmation {
cmd!(sh, "sudo rm -rf /usr/lib/ckan").run()?;
cmd!(sh, "sudo rm -rf /etc/ckan").run()?;
sh.change_dir(format!("/home/{username}"));
cmd!(sh, "rm -rf qsv*").run()?;
cmd!(sh, "rm -rf README ckan-compose ahoy dpp_default_config.ini get-docker.sh permissions.sql").run()?;
} else {
println!("Cancelling command.");
}
return Ok(());
}
let default_sysadmin = Sysadmin {
username: username.clone(),
password: "password".to_string(),
email: format!("{username}@localhost"),
};
let config = Config {
ssh: args
.features
.is_some_and(|features| features.contains(&"enable-ssh".to_string())),
ckan_version: if args.ckan_version.is_some() {
args.ckan_version.unwrap()
} else {
"2.11.3".to_string()
},
sysadmin: default_sysadmin.clone(),
extension_datastore: args
.extensions
.clone()
.is_some_and(|extensions| extensions.contains(&"DataStore".to_string())),
extension_ckanext_scheming: args
.extensions
.clone()
.is_some_and(|extensions| extensions.contains(&"ckanext-scheming".to_string())),
extension_datapusher_plus: args
.extensions
.is_some_and(|extensions| extensions.contains(&"DataPusher+".to_string())),
druf_mode: false,
};
steps::step_intro();
let mut default_config_text =
String::from("The current configuration for ckan-devstaller does the following:");
if config.ssh {
default_config_text.push_str("\n- Install openssh-server to enable SSH access");
}
default_config_text.push_str("\n- Install ckan-compose (https://github.com/tino097/ckan-compose) which sets up the CKAN backend (PostgreSQL, SOLR, Redis)");
default_config_text.push_str(format!("\n- Install CKAN v{}", config.ckan_version).as_str());
if config.extension_datastore {
default_config_text.push_str("\n- Install the DataStore extension");
}
if config.extension_ckanext_scheming {
default_config_text.push_str("\n- Install the ckanext-scheming extension");
}
if config.extension_datapusher_plus {
default_config_text.push_str("\n- Install the DataPusher+ extension");
default_config_text.push_str("\n- Disable DRUF mode for DataPusher+");
}
println!("{default_config_text}");
let answer_customize = if args.skip_interactive {
false
} else {
Confirm::new("Would you like to customize the configuration for your CKAN installation?")
.prompt()?
};
let config = if answer_customize {
let answer_ssh = question_ssh()?;
let answer_ckan_version = question_ckan_version()?;
@ -107,109 +184,33 @@ fn main() -> Result<()> {
druf_mode: answer_druf_mode,
}
} else {
Config {
ssh: true,
ckan_version: "2.11.3".to_string(),
sysadmin: default_sysadmin,
extension_datastore: true,
extension_ckanext_scheming: true,
extension_datapusher_plus: true,
druf_mode: false,
}
config
};
let begin_installation = if args.default {
let begin_installation = if args.skip_interactive {
true
} else {
Confirm::new("Would you like to begin the installation?").prompt()?
};
if begin_installation {
println!("{}", important_text("Starting installation..."));
println!(
"\n{} Running {} and {}...",
step_text("1."),
highlighted_text("sudo apt update -y"),
highlighted_text("sudo apt upgrade -y")
);
println!(
"{}",
important_text("You may need to provide your sudo password.")
);
cmd!(sh, "sudo apt update -y").run()?;
// Ignoring xrdp error with .ignore_status() for now
cmd!(sh, "sudo apt upgrade -y").ignore_status().run()?;
println!(
"{}",
success_text("✅ 1. Successfully ran update and upgrade commands.")
);
println!("\n{}", important_text("Starting installation..."));
// Run sudo apt update and sudo apt upgrade
step_package_updates("1.".to_string(), &sh)?;
println!(
"\n{} Installing {}...",
step_text("2."),
highlighted_text("curl")
);
cmd!(sh, "sudo apt install curl -y").run()?;
println!("{}", success_text("✅ 2.1. Successfully installed curl."));
// Install curl
step_install_curl("2.".to_string(), &sh)?;
// If user wants SSH capability, install openssh-server
if config.ssh {
println!("\n{} Installing openssh-server...", step_text("2."));
cmd!(sh, "sudo apt install openssh-server -y").run()?;
}
println!(
"{}",
success_text("✅ 2.2. Successfully installed openssh-server.")
);
let dpkg_l_output = cmd!(sh, "dpkg -l").read()?;
let has_docker = cmd!(sh, "grep docker")
.stdin(dpkg_l_output.clone())
.ignore_status()
.output()?
.status
.success();
if !has_docker {
println!("{} Installing Docker...", step_text("3."),);
cmd!(
sh,
"curl -fsSL https://get.docker.com -o /home/{username}/get-docker.sh"
)
.run()?;
cmd!(sh, "sudo sh /home/{username}/get-docker.sh").run()?;
println!("{}", success_text("✅ 3. Successfully installed Docker."));
step_install_openssh("2.".to_string(), &sh)?;
}
let has_docker_compose = cmd!(sh, "grep docker-compose")
.stdin(dpkg_l_output)
.ignore_status()
.output()?
.status
.success();
if !has_docker_compose {
cmd!(sh, "sudo apt install docker-compose -y").run()?;
}
// Install docker CLI if user does not have it installed
step_install_docker("3.".to_string(), &sh, username.clone())?;
println!("\n{} Installing Ahoy...", step_text("4."),);
sh.change_dir(format!("/home/{username}"));
cmd!(sh, "sudo curl -LO https://github.com/ahoy-cli/ahoy/releases/download/v2.5.0/ahoy-bin-linux-amd64").run()?;
cmd!(sh, "mv ./ahoy-bin-linux-amd64 ./ahoy").run()?;
cmd!(sh, "sudo chmod +x ./ahoy").run()?;
println!("{}", success_text("✅ 4. Successfully installed Ahoy."));
step_install_ahoy("4.".to_string(), &sh, username.clone())?;
println!(
"\n{} Downloading, installing, and starting ckan-compose...",
step_text("5."),
);
if !std::fs::exists(format!("/home/{username}/ckan-compose"))? {
cmd!(sh, "git clone https://github.com/tino097/ckan-compose.git").run()?;
}
sh.change_dir(format!("/home/{username}/ckan-compose"));
cmd!(sh, "git switch ckan-devstaller").run()?;
let env_data = "PROJECT_NAME=ckan-devstaller-project
DATASTORE_READONLY_PASSWORD=pass
POSTGRES_PASSWORD=pass";
std::fs::write(format!("/home/{username}/ckan-compose/.env"), env_data)?;
cmd!(sh, "sudo ../ahoy up").run()?;
println!("{}", success_text("✅ 5. Successfully ran ckan-compose."));
step_install_and_run_ckan_compose("5.".to_string(), &sh, username.clone())?;
println!(
"\n{} Installing CKAN {}...",
@ -260,210 +261,29 @@ POSTGRES_PASSWORD=pass";
.run()?;
println!(
"{}",
success_text(format!("6. Installed CKAN {}.", config.ckan_version).as_str())
success_text(format!("6. Installed CKAN {}.", config.ckan_version).as_str())
);
// Install extensions
if config.extension_datastore {
step_install_datastore_extension("7.".to_string(), &sh, username.clone())?;
}
if config.extension_ckanext_scheming {
step_install_ckanext_scheming_extension("8.".to_string(), &sh, username.clone())?;
}
if config.extension_datapusher_plus {
println!(
"\n{} Enabling DataStore plugin, adding config URLs in /etc/ckan/default/ckan.ini and updating permissions...",
step_text("7."),
);
let mut conf = ini::Ini::load_from_file("/etc/ckan/default/ckan.ini")?;
let app_main_section = conf.section_mut(Some("app:main")).unwrap();
let mut ckan_plugins = app_main_section.get("ckan.plugins").unwrap().to_string();
ckan_plugins.push_str(" datastore");
app_main_section.insert("ckan.plugins", ckan_plugins);
app_main_section.insert(
"ckan.datastore.write_url",
"postgresql://ckan_default:pass@localhost/datastore_default",
);
app_main_section.insert(
"ckan.datastore.read_url",
"postgresql://datastore_default:pass@localhost/datastore_default",
);
app_main_section.insert("ckan.datastore.sqlsearch.enabled", "true");
conf.write_to_file("/etc/ckan/default/ckan.ini")?;
let postgres_container_id = cmd!(
sh,
"sudo docker ps -aqf name=^ckan-devstaller-project-postgres$"
)
.read()?;
let set_permissions_output = cmd!(
sh,
"ckan -c /etc/ckan/default/ckan.ini datastore set-permissions"
)
.read()?;
std::fs::write("permissions.sql", set_permissions_output)?;
loop {
std::thread::sleep(std::time::Duration::from_secs(2));
if std::fs::exists("permissions.sql")? {
break;
}
}
sh.change_dir(format!("/home/{username}"));
cmd!(
sh,
"sudo docker cp permissions.sql {postgres_container_id}:/permissions.sql"
)
.run()?;
cmd!(sh, "sudo docker exec {postgres_container_id} psql -U ckan_default --set ON_ERROR_STOP=1 -f permissions.sql").run()?;
println!(
"{}",
success_text(
"✅ 7. Enabled DataStore plugin, set DataStore URLs in /etc/ckan/default/ckan.ini, and updated permissions."
)
);
println!(
"{}",
step_text("\n{} Installing ckanext-scheming and DataPusher+ extensions..."),
);
cmd!(
sh,
"pip install -e git+https://github.com/ckan/ckanext-scheming.git#egg=ckanext-scheming"
)
.run()?;
let mut conf = ini::Ini::load_from_file("/etc/ckan/default/ckan.ini")?;
let app_main_section = conf.section_mut(Some("app:main")).unwrap();
let mut ckan_plugins = app_main_section.get("ckan.plugins").unwrap().to_string();
ckan_plugins.push_str(" scheming_datasets");
cmd!(
sh,
"ckan config-tool /etc/ckan/default/ckan.ini -s app:main ckan.plugins={ckan_plugins}"
)
.run()?;
cmd!(sh, "ckan config-tool /etc/ckan/default/ckan.ini -s app:main scheming.presets=ckanext.scheming:presets.json").run()?;
cmd!(sh, "ckan config-tool /etc/ckan/default/ckan.ini -s app:main scheming.dataset_fallback=false").run()?;
// app_main_section.insert("ckan.plugins", ckan_plugins);
// app_main_section.insert("scheming.presets", "ckanext.scheming:presets.json");
// app_main_section.insert("scheming.dataset_fallback", "false");
// conf.write_to_file("/etc/ckan/default/ckan.ini")?;
// Install DataPusher+
cmd!(sh, "sudo apt install python3-virtualenv python3-dev python3-pip python3-wheel build-essential libxslt1-dev libxml2-dev zlib1g-dev git libffi-dev libpq-dev uchardet -y").run()?;
sh.change_dir("/usr/lib/ckan/default/src");
cmd!(sh, "pip install -e git+https://github.com/dathere/datapusher-plus.git@main#egg=datapusher-plus").run()?;
sh.change_dir("/usr/lib/ckan/default/src/datapusher-plus");
cmd!(sh, "pip install -r requirements.txt").run()?;
sh.change_dir(format!("/home/{username}"));
cmd!(sh, "wget https://github.com/dathere/qsv/releases/download/4.0.0/qsv-4.0.0-x86_64-unknown-linux-gnu.zip").run()?;
cmd!(sh, "sudo apt install unzip -y").run()?;
cmd!(sh, "unzip qsv-4.0.0-x86_64-unknown-linux-gnu.zip").run()?;
cmd!(sh, "sudo rm -rf qsv-4.0.0-x86_64-unknown-linux-gnu.zip").run()?;
cmd!(sh, "sudo mv ./qsvdp_glibc-2.31 /usr/local/bin/qsvdp").run()?;
let mut conf = ini::Ini::load_from_file("/etc/ckan/default/ckan.ini")?;
let app_main_section = conf.section_mut(Some("app:main")).unwrap();
let mut ckan_plugins = app_main_section.get("ckan.plugins").unwrap().to_string();
ckan_plugins.push_str(" datapusher_plus");
cmd!(
sh,
"ckan config-tool /etc/ckan/default/ckan.ini -s app:main ckan.plugins={ckan_plugins}"
)
.run()?;
cmd!(sh, "ckan config-tool /etc/ckan/default/ckan.ini -s app:main scheming.dataset_schemas=ckanext.datapusher_plus:dataset-druf.yaml").run()?;
// app_main_section.insert("ckan.plugins", ckan_plugins);
// app_main_section.insert(
// "scheming.dataset_schemas",
// "ckanext.datapusher_plus:dataset-druf.yaml",
// );
// conf.write_to_file("/etc/ckan/default/ckan.ini")?;
let dpp_default_config = r#"
ckanext.datapusher_plus.use_proxy = false
ckanext.datapusher_plus.download_proxy =
ckanext.datapusher_plus.ssl_verify = false
# supports INFO, DEBUG, TRACE - use DEBUG or TRACE when debugging scheming Formulas
ckanext.datapusher_plus.upload_log_level = INFO
ckanext.datapusher_plus.formats = csv tsv tab ssv xls xlsx xlsxb xlsm ods geojson shp qgis zip
ckanext.datapusher_plus.pii_screening = false
ckanext.datapusher_plus.pii_found_abort = false
ckanext.datapusher_plus.pii_regex_resource_id_or_alias =
ckanext.datapusher_plus.pii_show_candidates = false
ckanext.datapusher_plus.pii_quick_screen = false
ckanext.datapusher_plus.qsv_bin = /usr/local/bin/qsvdp
ckanext.datapusher_plus.preview_rows = 100
ckanext.datapusher_plus.download_timeout = 300
ckanext.datapusher_plus.max_content_length = 1256000000000
ckanext.datapusher_plus.chunk_size = 16384
ckanext.datapusher_plus.default_excel_sheet = 0
ckanext.datapusher_plus.sort_and_dupe_check = true
ckanext.datapusher_plus.dedup = false
ckanext.datapusher_plus.unsafe_prefix = unsafe_
ckanext.datapusher_plus.reserved_colnames = _id
ckanext.datapusher_plus.prefer_dmy = false
ckanext.datapusher_plus.ignore_file_hash = true
ckanext.datapusher_plus.auto_index_threshold = 3
ckanext.datapusher_plus.auto_index_dates = true
ckanext.datapusher_plus.auto_unique_index = true
ckanext.datapusher_plus.summary_stats_options =
ckanext.datapusher_plus.add_summary_stats_resource = false
ckanext.datapusher_plus.summary_stats_with_preview = false
ckanext.datapusher_plus.qsv_stats_string_max_length = 32767
ckanext.datapusher_plus.qsv_dates_whitelist = date,time,due,open,close,created
ckanext.datapusher_plus.qsv_freq_limit = 10
ckanext.datapusher_plus.auto_alias = true
ckanext.datapusher_plus.auto_alias_unique = false
ckanext.datapusher_plus.copy_readbuffer_size = 1048576
ckanext.datapusher_plus.type_mapping = {"String": "text", "Integer": "numeric","Float": "numeric","DateTime": "timestamp","Date": "date","NULL": "text"}
ckanext.datapusher_plus.auto_spatial_simplication = true
ckanext.datapusher_plus.spatial_simplication_relative_tolerance = 0.1
ckanext.datapusher_plus.latitude_fields = latitude,lat
ckanext.datapusher_plus.longitude_fields = longitude,long,lon
ckanext.datapusher_plus.jinja2_bytecode_cache_dir = /tmp/jinja2_butecode_cache
ckanext.datapusher_plus.auto_unzip_one_file = true
ckanext.datapusher_plus.api_token = <CKAN service account token for CKAN user with sysadmin privileges>
ckanext.datapusher_plus.describeGPT_api_key = <Token for OpenAI API compatible service>
ckanext.datapusher_plus.file_bin = /usr/bin/file
ckanext.datapusher_plus.enable_druf = false
ckanext.datapusher_plus.enable_form_redirect = true
"#;
std::fs::write("dpp_default_config.ini", dpp_default_config)?;
cmd!(
sh,
"ckan config-tool /etc/ckan/default/ckan.ini -f dpp_default_config.ini"
)
.run()?;
let resource_formats_str = std::fs::read_to_string(
"/usr/lib/ckan/default/src/ckan/config/resource_formats.json",
step_install_datapusher_plus_extension(
"9.".to_string(),
&sh,
sysadmin_username,
username.clone(),
)?;
let mut resource_formats_val: serde_json::Value =
serde_json::from_str(&resource_formats_str)?;
let all_resource_formats = resource_formats_val
.get_mut(0)
.unwrap()
.as_array_mut()
.unwrap();
all_resource_formats.push(json!([
"TAB",
"Tab Separated Values File",
"text/tab-separated-values",
[]
]));
std::fs::write(
"/usr/lib/ckan/default/src/ckan/config/resource_formats.json",
serde_json::to_string(&resource_formats_val)?,
)?;
cmd!(sh, "sudo locale-gen en_US.UTF-8").run()?;
cmd!(sh, "sudo update-locale").run()?;
let token_command_output = cmd!(
sh,
"ckan -c /etc/ckan/default/ckan.ini user token add {sysadmin_username} dpplus"
)
.read()?;
let tail_output = cmd!(sh, "tail -n 1").stdin(token_command_output).read()?;
let dpp_api_token = cmd!(sh, "tr -d '\t'").stdin(tail_output).read()?;
cmd!(sh, "ckan config-tool /etc/ckan/default/ckan.ini ckanext.datapusher_plus.api_token={dpp_api_token}").env("LC_ALL", "en_US.UTF-8").run()?;
cmd!(
sh,
"ckan -c /etc/ckan/default/ckan.ini db upgrade -p datapusher_plus"
)
.run()?;
println!(
"{}",
success_text("✅ 8. Installed ckanext-scheming and DataPusher+ extensions.")
);
}
println!("\n{}", success_text("Running CKAN instance..."));
println!("\n{}", success_text("Running CKAN instance..."));
cmd!(sh, "ckan -c /etc/ckan/default/ckan.ini run").run()?;
} else {
println!("Cancelling installation.");
}
Ok(())

View file

@ -1,4 +1,7 @@
use crate::styles::{highlighted_text, important_text};
use crate::styles::{highlighted_text, important_text, step_text, success_text};
use anyhow::Result;
use serde_json::json;
use xshell::{Shell, cmd};
pub fn step_intro() {
println!("Welcome to the ckan-devstaller!");
@ -11,9 +14,374 @@ pub fn step_intro() {
highlighted_text("CKAN 2.11.3")
);
println!(
"{}",
"\nYou may also learn more about ckan-devstaller at https://ckan-devstaller.dathere.com."
);
println!(
"\n{}\n",
important_text(
"This installer is only intended for a brand new installation of Ubuntu 22.04."
)
);
}
pub fn step_package_updates(step_prefix: String, sh: &Shell) -> Result<()> {
println!(
"\n{} Running {} and {}...",
step_text(step_prefix.as_str()),
highlighted_text("sudo apt update -y"),
highlighted_text("sudo apt upgrade -y")
);
println!(
"{}",
important_text("You may need to provide your sudo password.")
);
cmd!(sh, "sudo apt update -y").run()?;
// Ignoring xrdp error with .ignore_status() for now
cmd!(sh, "sudo apt upgrade -y").ignore_status().run()?;
println!(
"{}",
success_text(
format!("{step_prefix} Successfully ran update and upgrade commands.").as_str()
)
);
Ok(())
}
pub fn step_install_curl(step_prefix: String, sh: &Shell) -> Result<()> {
println!(
"\n{} Installing {}...",
step_text("2."),
highlighted_text("curl")
);
cmd!(sh, "sudo apt install curl -y").run()?;
println!(
"{}",
success_text(format!("{step_prefix} Successfully installed curl.").as_str())
);
Ok(())
}
pub fn step_install_openssh(step_prefix: String, sh: &Shell) -> Result<()> {
println!(
"\n{} Installing openssh-server...",
step_text(step_prefix.as_str())
);
cmd!(sh, "sudo apt install openssh-server -y").run()?;
println!(
"{}",
success_text(format!("{step_prefix} Successfully installed openssh-server.").as_str())
);
Ok(())
}
pub fn step_install_docker(step_prefix: String, sh: &Shell, username: String) -> Result<()> {
let dpkg_l_output = cmd!(sh, "dpkg -l").read()?;
let has_docker = cmd!(sh, "grep docker")
.stdin(dpkg_l_output.clone())
.ignore_status()
.output()?
.status
.success();
if !has_docker {
println!("{} Installing Docker...", step_text(step_prefix.as_str()),);
cmd!(
sh,
"curl -fsSL https://get.docker.com -o /home/{username}/get-docker.sh"
)
.run()?;
cmd!(sh, "sudo sh /home/{username}/get-docker.sh").run()?;
println!(
"{}",
success_text(format!("{step_prefix} Successfully installed Docker.").as_str())
);
}
Ok(())
}
pub fn step_install_ahoy(step_prefix: String, sh: &Shell, username: String) -> Result<()> {
println!("\n{} Installing Ahoy...", step_text(step_prefix.as_str()),);
sh.change_dir(format!("/home/{username}"));
cmd!(sh, "sudo curl -LO https://github.com/ahoy-cli/ahoy/releases/download/v2.5.0/ahoy-bin-linux-amd64").run()?;
cmd!(sh, "mv ./ahoy-bin-linux-amd64 ./ahoy").run()?;
cmd!(sh, "sudo chmod +x ./ahoy").run()?;
println!(
"{}",
success_text(format!("{step_prefix} Successfully installed Ahoy.").as_str())
);
Ok(())
}
pub fn step_install_and_run_ckan_compose(
step_prefix: String,
sh: &Shell,
username: String,
) -> Result<()> {
println!(
"\n{} Downloading, installing, and starting ckan-compose...",
step_text(step_prefix.as_str()),
);
if !std::fs::exists(format!("/home/{username}/ckan-compose"))? {
cmd!(sh, "git clone https://github.com/tino097/ckan-compose.git").run()?;
}
sh.change_dir(format!("/home/{username}/ckan-compose"));
cmd!(sh, "git switch ckan-devstaller").run()?;
let env_data = "PROJECT_NAME=ckan-devstaller-project
DATASTORE_READONLY_PASSWORD=pass
POSTGRES_PASSWORD=pass";
std::fs::write(format!("/home/{username}/ckan-compose/.env"), env_data)?;
cmd!(sh, "sudo ../ahoy up").run()?;
println!(
"{}",
success_text(format!("{step_prefix} Successfully ran ckan-compose.").as_str())
);
Ok(())
}
pub fn step_install_datastore_extension(
step_prefix: String,
sh: &Shell,
username: String,
) -> Result<()> {
println!(
"\n{} Enabling DataStore plugin, adding config URLs in /etc/ckan/default/ckan.ini and updating permissions...",
step_text(step_prefix.as_str()),
);
let mut conf = ini::Ini::load_from_file("/etc/ckan/default/ckan.ini")?;
let app_main_section = conf.section_mut(Some("app:main")).unwrap();
let mut ckan_plugins = app_main_section.get("ckan.plugins").unwrap().to_string();
ckan_plugins.push_str(" datastore");
app_main_section.insert("ckan.plugins", ckan_plugins);
app_main_section.insert(
"ckan.datastore.write_url",
"postgresql://ckan_default:pass@localhost/datastore_default",
);
app_main_section.insert(
"ckan.datastore.read_url",
"postgresql://datastore_default:pass@localhost/datastore_default",
);
app_main_section.insert("ckan.datastore.sqlsearch.enabled", "true");
conf.write_to_file("/etc/ckan/default/ckan.ini")?;
let postgres_container_id = cmd!(
sh,
"sudo docker ps -aqf name=^ckan-devstaller-project-postgres$"
)
.read()?;
let set_permissions_output = cmd!(
sh,
"ckan -c /etc/ckan/default/ckan.ini datastore set-permissions"
)
.read()?;
std::fs::write("permissions.sql", set_permissions_output)?;
loop {
std::thread::sleep(std::time::Duration::from_secs(2));
if std::fs::exists("permissions.sql")? {
break;
}
}
sh.change_dir(format!("/home/{username}"));
cmd!(
sh,
"sudo docker cp permissions.sql {postgres_container_id}:/permissions.sql"
)
.run()?;
cmd!(sh, "sudo docker exec {postgres_container_id} psql -U ckan_default --set ON_ERROR_STOP=1 -f permissions.sql").run()?;
println!(
"{}",
success_text(
format!("{step_prefix} Enabled DataStore plugin, set DataStore URLs in /etc/ckan/default/ckan.ini, and updated permissions.").as_str()
)
);
Ok(())
}
pub fn step_install_ckanext_scheming_extension(
step_prefix: String,
sh: &Shell,
username: String,
) -> Result<()> {
println!(
"{}",
step_text("\n{} Installing the ckanext-scheming extension..."),
);
cmd!(
sh,
"pip install -e git+https://github.com/ckan/ckanext-scheming.git#egg=ckanext-scheming"
)
.run()?;
let mut conf = ini::Ini::load_from_file("/etc/ckan/default/ckan.ini")?;
let app_main_section = conf.section_mut(Some("app:main")).unwrap();
let mut ckan_plugins = app_main_section.get("ckan.plugins").unwrap().to_string();
ckan_plugins.push_str(" scheming_datasets");
cmd!(
sh,
"ckan config-tool /etc/ckan/default/ckan.ini -s app:main ckan.plugins={ckan_plugins}"
)
.run()?;
cmd!(sh, "ckan config-tool /etc/ckan/default/ckan.ini -s app:main scheming.presets=ckanext.scheming:presets.json").run()?;
cmd!(
sh,
"ckan config-tool /etc/ckan/default/ckan.ini -s app:main scheming.dataset_fallback=false"
)
.run()?;
// app_main_section.insert("ckan.plugins", ckan_plugins);
// app_main_section.insert("scheming.presets", "ckanext.scheming:presets.json");
// app_main_section.insert("scheming.dataset_fallback", "false");
// conf.write_to_file("/etc/ckan/default/ckan.ini")?;
Ok(())
}
pub fn step_install_datapusher_plus_extension(
step_prefix: String,
sh: &Shell,
sysadmin_username: String,
username: String,
) -> Result<()> {
// Install DataPusher+
println!(
"{}",
step_text(format!("\n{step_prefix} Installing DataPusher+ extension...").as_str())
);
cmd!(sh, "sudo apt install python3-virtualenv python3-dev python3-pip python3-wheel build-essential libxslt1-dev libxml2-dev zlib1g-dev git libffi-dev libpq-dev uchardet -y").run()?;
sh.change_dir("/usr/lib/ckan/default/src");
cmd!(
sh,
"pip install -e git+https://github.com/dathere/datapusher-plus.git@main#egg=datapusher-plus"
)
.run()?;
sh.change_dir("/usr/lib/ckan/default/src/datapusher-plus");
cmd!(sh, "pip install -r requirements.txt").run()?;
sh.change_dir(format!("/home/{username}"));
cmd!(sh, "wget https://github.com/dathere/qsv/releases/download/4.0.0/qsv-4.0.0-x86_64-unknown-linux-gnu.zip").run()?;
cmd!(sh, "sudo apt install unzip -y").run()?;
cmd!(sh, "unzip qsv-4.0.0-x86_64-unknown-linux-gnu.zip").run()?;
cmd!(sh, "sudo rm -rf qsv-4.0.0-x86_64-unknown-linux-gnu.zip").run()?;
cmd!(sh, "sudo mv ./qsvdp_glibc-2.31 /usr/local/bin/qsvdp").run()?;
let mut conf = ini::Ini::load_from_file("/etc/ckan/default/ckan.ini")?;
let app_main_section = conf.section_mut(Some("app:main")).unwrap();
let mut ckan_plugins = app_main_section.get("ckan.plugins").unwrap().to_string();
ckan_plugins.push_str(" datapusher_plus");
app_main_section.insert("ckan.plugins", ckan_plugins);
app_main_section.insert(
"scheming.dataset_schemas",
"ckanext.datapusher_plus:dataset-druf.yaml",
);
app_main_section.insert("ckanext.datapusher_plus.use_proxy", "false");
app_main_section.insert("ckanext.datapusher_plus.download_proxy", "");
app_main_section.insert("ckanext.datapusher_plus.ssl_verify", "false");
app_main_section.insert("ckanext.datapusher_plus.upload_log_level", "INFO");
app_main_section.insert(
"ckanext.datapusher_plus.formats",
"csv tsv tab ssv xls xlsx xlsxb xlsm ods geojson shp qgis zip",
);
app_main_section.insert("ckanext.datapusher_plus.pii_screening", "false");
app_main_section.insert("ckanext.datapusher_plus.pii_found_abort", "false");
app_main_section.insert("ckanext.datapusher_plus.pii_regex_resource_id_or_alias", "");
app_main_section.insert("ckanext.datapusher_plus.pii_show_candidates", "false");
app_main_section.insert("ckanext.datapusher_plus.pii_quick_screen", "false");
app_main_section.insert("ckanext.datapusher_plus.qsv_bin", "/usr/local/bin/qsvdp");
app_main_section.insert("ckanext.datapusher_plus.preview_rows", "100");
app_main_section.insert("ckanext.datapusher_plus.download_timeout", "300");
app_main_section.insert(
"ckanext.datapusher_plus.max_content_length",
"1256000000000",
);
app_main_section.insert("ckanext.datapusher_plus.chunk_size", "16384");
app_main_section.insert("ckanext.datapusher_plus.default_excel_sheet", "0");
app_main_section.insert("ckanext.datapusher_plus.sort_and_dupe_check", "true");
app_main_section.insert("ckanext.datapusher_plus.dedup", "false");
app_main_section.insert("ckanext.datapusher_plus.unsafe_prefix", "unsafe_");
app_main_section.insert("ckanext.datapusher_plus.reserved_colnames", "_id");
app_main_section.insert("ckanext.datapusher_plus.prefer_dmy", "false");
app_main_section.insert("ckanext.datapusher_plus.ignore_file_hash", "true");
app_main_section.insert("ckanext.datapusher_plus.auto_index_threshold", "3");
app_main_section.insert("ckanext.datapusher_plus.auto_index_dates", "true");
app_main_section.insert("ckanext.datapusher_plus.auto_unique_index", "true");
app_main_section.insert("ckanext.datapusher_plus.summary_stats_options", "");
app_main_section.insert(
"ckanext.datapusher_plus.add_summary_stats_resource",
"false",
);
app_main_section.insert(
"ckanext.datapusher_plus.summary_stats_with_preview",
"false",
);
app_main_section.insert(
"ckanext.datapusher_plus.qsv_stats_string_max_length",
"32767",
);
app_main_section.insert(
"ckanext.datapusher_plus.qsv_dates_whitelist",
"date,time,due,open,close,created",
);
app_main_section.insert("ckanext.datapusher_plus.qsv_freq_limit", "10");
app_main_section.insert("ckanext.datapusher_plus.auto_alias", "true");
app_main_section.insert("ckanext.datapusher_plus.auto_alias_unique", "false");
app_main_section.insert("ckanext.datapusher_plus.copy_readbuffer_size", "1048576");
app_main_section.insert("ckanext.datapusher_plus.type_mapping", r#"{"String": "text", "Integer": "numeric","Float": "numeric","DateTime": "timestamp","Date": "date","NULL": "text"}"#);
app_main_section.insert("ckanext.datapusher_plus.auto_spatial_simplication", "true");
app_main_section.insert(
"ckanext.datapusher_plus.spatial_simplication_relative_tolerance",
"0.1",
);
app_main_section.insert("ckanext.datapusher_plus.latitude_fields", "latitude,lat");
app_main_section.insert(
"ckanext.datapusher_plus.longitude_fields",
"longitude,long,lon",
);
app_main_section.insert(
"ckanext.datapusher_plus.jinja2_bytecode_cache_dir",
"/tmp/jinja2_butecode_cache",
);
app_main_section.insert("ckanext.datapusher_plus.auto_unzip_one_file", "true");
app_main_section.insert(
"ckanext.datapusher_plus.api_token",
"<CKAN service account token for CKAN user with sysadmin privileges>",
);
app_main_section.insert(
"ckanext.datapusher_plus.describeGPT_api_key",
"<Token for OpenAI API compatible service>",
);
app_main_section.insert("ckanext.datapusher_plus.file_bin", "/usr/bin/file");
app_main_section.insert("ckanext.datapusher_plus.enable_druf", "false");
app_main_section.insert("ckanext.datapusher_plus.enable_form_redirect", "true");
conf.write_to_file("/etc/ckan/default/ckan.ini")?;
let resource_formats_str = std::fs::read_to_string(
"/usr/lib/ckan/default/src/ckan/ckan/config/resource_formats.json",
)?;
let mut resource_formats_val: serde_json::Value = serde_json::from_str(&resource_formats_str)?;
let all_resource_formats = resource_formats_val
.get_mut(0)
.unwrap()
.as_array_mut()
.unwrap();
all_resource_formats.push(json!([
"TAB",
"Tab Separated Values File",
"text/tab-separated-values",
[]
]));
std::fs::write(
"/usr/lib/ckan/default/src/ckan/ckan/config/resource_formats.json",
serde_json::to_string(&resource_formats_val)?,
)?;
cmd!(sh, "sudo locale-gen en_US.UTF-8").run()?;
cmd!(sh, "sudo update-locale").run()?;
let token_command_output = cmd!(
sh,
"ckan -c /etc/ckan/default/ckan.ini user token add {sysadmin_username} dpplus"
)
.read()?;
let tail_output = cmd!(sh, "tail -n 1").stdin(token_command_output).read()?;
let dpp_api_token = cmd!(sh, "tr -d '\t'").stdin(tail_output).read()?;
cmd!(sh, "ckan config-tool /etc/ckan/default/ckan.ini ckanext.datapusher_plus.api_token={dpp_api_token}").env("LC_ALL", "en_US.UTF-8").run()?;
cmd!(
sh,
"ckan -c /etc/ckan/default/ckan.ini db upgrade -p datapusher_plus"
)
.run()?;
println!(
"{}",
success_text(format!("{step_prefix} Installed DataPusher+ extension.").as_str())
);
Ok(())
}