This commit is contained in:
atagen 2026-05-19 23:32:51 +10:00
parent da4bc139eb
commit 69a5ff67e1
49 changed files with 8319 additions and 5483 deletions

View file

@ -1,141 +1,77 @@
# building and installing
## dependencies
inshellah is a rust crate. it builds with stock cargo on any platform
rust supports.
inshellah is written in OCaml and uses dune as its build system.
build dependencies:
- **OCaml** >= 5.0
- **dune** >= 3.20
- **angstrom** — parser combinator library
- **angstrom-unix** — unix extensions for angstrom
- **camlzip** — gzip decompression for reading compressed manpages
- **str** — regular expressions (ships with OCaml)
- **unix** — process/file operations (ships with OCaml)
runtime dependencies:
- **man** (optional) — used as a fallback to locate manpages during
on-the-fly completion resolution. not needed if system directories
are provided via `--dir` (manpages are found via sibling `share/man`).
## building with nix (recommended)
if you have nix installed:
## with nix
```sh
nix build
```
the binary is at `./result/bin/inshellah`.
binary is at `./result/bin/inshellah`.
for development with a shell containing all dependencies:
development shell:
```sh
nix develop
dune build
dune test
cargo build --release
cargo test
```
## building from source with opam
## with cargo
install dependencies via opam:
requires rust >= 1.85 (edition 2024).
```sh
opam install dune angstrom angstrom-unix camlzip
```
build and test:
```sh
dune build
dune test
```
install into the opam switch:
```sh
dune install
```
## building from source without opam
if your distribution packages the OCaml libraries directly, install
them through your package manager, then build with dune:
```sh
dune build
```
the binary is at `_build/default/bin/main.exe`. copy it to your
`$PATH`:
```sh
install -Dm755 _build/default/bin/main.exe /usr/local/bin/inshellah
cargo build --release
cargo test
sudo install -Dm755 target/release/inshellah /usr/local/bin/inshellah
```
## arch linux
install OCaml and dune from the official repos, and the remaining
libraries from the AUR or via opam:
```sh
# system packages
sudo pacman -S ocaml dune
# ocaml libraries (via opam)
opam init # if not already initialized
eval $(opam env)
opam install angstrom angstrom-unix camlzip
# build
dune build
dune test
# install
sudo install -Dm755 _build/default/bin/main.exe /usr/local/bin/inshellah
sudo pacman -S rust
cargo build --release
sudo install -Dm755 target/release/inshellah /usr/local/bin/inshellah
```
## debian / ubuntu
```sh
sudo apt install ocaml opam
opam init
eval $(opam env)
opam install dune angstrom angstrom-unix camlzip
dune build
sudo install -Dm755 _build/default/bin/main.exe /usr/local/bin/inshellah
sudo apt install cargo rustc
# or: rustup install stable
cargo build --release
sudo install -Dm755 target/release/inshellah /usr/local/bin/inshellah
```
## fedora
```sh
sudo dnf install ocaml opam
opam init
eval $(opam env)
opam install dune angstrom angstrom-unix camlzip
dune build
sudo install -Dm755 _build/default/bin/main.exe /usr/local/bin/inshellah
sudo dnf install cargo rust
cargo build --release
sudo install -Dm755 target/release/inshellah /usr/local/bin/inshellah
```
## post-install setup
after installing the binary, index completions from your system
prefix(es):
index completions from your system prefix(es):
```sh
# typical linux system
inshellah index /usr /usr/local
# more workers / different timeout
inshellah index /usr /usr/local --workers 16 --timeout-ms 500
# check what was indexed
inshellah dump
```
then wire up the nushell completer:
wire up the nushell completer in `~/.config/nushell/config.nu`:
```nu
# ~/.config/nushell/config.nu
$env.config.completions.external = {
enable: true
completer: {|spans|
@ -145,19 +81,28 @@ $env.config.completions.external = {
}
```
see [nushell-integration.md](nushell-integration.md) for full details
on the completer, and [runtime-completions.md](runtime-completions.md)
for on-the-fly resolution of commands not covered by the index.
see [nushell-integration.md](nushell-integration.md) for full
completer details and [runtime-completions.md](runtime-completions.md)
for on-the-fly resolution of commands not covered by the upfront
index.
## re-indexing after package changes
the index is a static cache — it doesn't update automatically when you
install or remove packages. re-run `inshellah index` after significant
package changes:
```sh
inshellah index /usr /usr/local
```
on nixos, the system index regenerates on every `nixos-rebuild`
automatically. see [nixos.md](nixos.md) for details.
on nixos, the system index regenerates on every `nixos-rebuild`. see
[nixos.md](nixos.md).
## development
```sh
cargo build # debug build, faster compile
cargo test # full test suite
cargo clippy --release
```
a `man` binary is useful at runtime as a fallback for locating
manpages outside the indexed prefixes — not required for indexing
itself.

View file

@ -1,105 +1,47 @@
# nixos integration
inshellah provides a nixos module that automatically indexes nushell
completions for all installed packages at system build time.
inshellah provides a nixos module that indexes nushell completions for
every installed package at system build time, and a wrapped binary
that knows where to find the result.
## enabling
```nix
# in your flake.nix outputs:
# flake.nix outputs:
{
nixosConfigurations.myhost = nixpkgs.lib.nixosSystem {
modules = [
inshellah.nixosModules.default
{
programs.inshellah.enable = true;
}
{ programs.inshellah.enable = true; }
];
};
}
```
or if importing the module directly:
or importing directly:
```nix
# configuration.nix
{ pkgs, ... }: {
imports = [ ./path/to/inshellah/nix/module.nix ];
programs.inshellah = {
enable = true;
package = pkgs.inshellah; # or your local build
};
imports = [ ./path/to/inshellah-rs/nix/module.nix ];
programs.inshellah.enable = true;
}
```
## what happens at build time
after rebuilding, completions are immediately available through the
autoloaded nushell shim.
the module hooks into `environment.extraSetup`, which runs during the
system profile build (the `buildEnv` that creates `/run/current-system/sw`).
at that point, all system packages are merged, so `$out/bin` contains every
executable and `$out/share/man` contains every manpage.
## what the module does
inshellah runs a single command:
```
inshellah index "$out" --dir $out/share/inshellah
```
this executes a three-phase pipeline:
### phase 1: native completion detection (parallel)
for each executable, inshellah scans the elf binary for the string
`completion`. if found, it probes common patterns like
`CMD completions nushell` to see if the program can generate its own
nushell completions. native output is used verbatim — these are always
higher quality than parsed completions.
programs like `niri`, and any clap/cobra tool with nushell support,
are handled this way.
### phase 2: manpage parsing (sequential)
for commands not covered by phase 1, inshellah parses manpages from
man1 (user commands) and man8 (sysadmin commands). it handles:
- gnu `.TP` style (coreutils, help2man)
- `.IP` style (curl, hand-written)
- `.PP`+`.RS`/`.RE` style (git, docbook)
- nix3 bullet+hyperlink style (`nix run`, `nix build`, etc.)
- mdoc (bsd) format
- deroff fallback for unusual formats
synopsis sections are parsed to detect subcommands: `git-commit.1`
generates `export extern "git commit"`, not `export extern "git-commit"`.
### phase 3: --help fallback (parallel)
remaining executables without manpages get `--help` (or `-h`) called
with a 200ms timeout. elf binaries are pre-scanned for the `-h` string
to skip those that don't support help flags. shell scripts are run
directly (they're fast). execution is parallelized to available cores.
when `--help` produces rendered manpage output instead of plain help
text (e.g. `git stash --help` delegates to `man`), the raw manpage
source is located and parsed with the groff parser for richer results.
### output
each command gets its own file in `/share/inshellah` under the system
profile. native generators produce `.nu` files; parsed results produce
`.json` files. the `complete` command reads both formats.
nushell built-in commands (ls, cd, cp, mv, etc.) are excluded since
nushell provides its own completions.
### performance
on a typical nixos system (~950 executables, ~1600 manpages):
- total time: ~4-10 seconds
- native gzip decompression (camlzip, no process spawning)
- parallel --help with core-scaled forking
- elf string scanning to skip ~15% of binaries
- installs the inshellah binary, wrapped so the system completion path
is found automatically.
- runs `inshellah index "$out"` during the system profile build,
producing one file per command under `$out/share/inshellah/`.
- drops the full nushell external-completer shim into
`/share/nushell/vendor/autoload/`, including sudo/doas overrides so
elevated commands still complete through inshellah.
- exposes the same shim as a read-only `snippet` option for users who
want to source or inspect it manually.
## module options
@ -110,12 +52,11 @@ programs.inshellah = {
# the inshellah package (set automatically by the flake module)
package = pkgs.inshellah;
# where to place indexed completion files under the system profile
# subdirectory of the system profile holding the index files
# default: "/share/inshellah"
completionsPath = "/share/inshellah";
# additional read-only completion directories to search
# these are appended to the --dir path alongside the system completions
extraDirs = [ "/etc/profiles/per-user/alice/share/inshellah" ];
# commands to skip entirely during indexing
@ -123,41 +64,68 @@ programs.inshellah = {
# commands to skip manpage parsing for (uses --help instead)
helpOnlyCommands = [ "nix" ];
# per-subprocess timeout in ms during indexing (null = built-in
# default of 200ms)
timeoutMs = null;
# worker-thread count for the parallel scrape
workers = null;
};
```
## using the completer
the flake module sets a read-only `snippet` option containing the nushell
config needed to wire up the completer. you can access it via
`config.programs.inshellah.snippet` and paste it into your nushell config,
or source it from a file generated by your nixos config.
the module installs the completer under nushell's vendor autoload path,
so no hand-written nushell config is needed for the normal NixOS case.
the snippet sets up the external completer. the wrapper installed by
the module has the system completion paths hardcoded, so no flags are
needed:
the read-only `snippet` option still holds the complete
external-completer config. to manage sourcing yourself instead of using
autoload, write it to a file:
```nu
let inshellah_complete = {|spans|
inshellah complete ...$spans | from json
}
$env.config.completions.external = {
enable: true
max_results: 100
completer: $inshellah_complete
}
```nix
# generate a config file from the snippet
environment.etc."nushell/inshellah.nu".text = config.programs.inshellah.snippet;
```
## home manager and other user-level package managers
then source that file from your nushell config:
the nixos module only indexes packages installed at the system level
(those that end up in `/run/current-system/sw`). if you use home-manager,
nix-env, or another user-level package manager, those binaries and
manpages live elsewhere — typically under `/etc/profiles/per-user/<name>`
or `~/.nix-profile`.
```nu
source /etc/nushell/inshellah.nu
```
to get completions for user-installed packages, run `inshellah index`
against those prefixes separately:
or copy the snippet directly into `~/.config/nushell/config.nu`:
```nu
# (the snippet is many lines — copy it from `nix eval` of the option,
# or use the environment.etc approach above)
$env.config.completions.external = { ... }
```
the snippet provides both static lookups against the system index and
runtime fallbacks for cases the static index can't cover:
| command | dynamic source |
|---|---|
| `nix` | flake refs via `NIX_GET_COMPLETIONS`, with optional `meta.description` |
| `systemctl` / `journalctl` | unit names from `list-units` |
| `coredumpctl` | units + pids |
| `loginctl` | users / sessions |
| `machinectl` / `networkctl` | machines / links |
| `ssh` / `scp` / `sftp` | hostnames from ssh config + known_hosts |
| `docker` / `podman` | containers + image refs by subcommand |
| `kubectl` | resource names from the live cluster |
| `git` | refs + worktree paths |
| `npm` / `pnpm` / `yarn` | scripts from package.json |
| `make` / `just` | targets / recipes |
| `cargo` | workspace targets behind `--bin` / `--example` / etc. |
| `kill` / `pkill` | pid+comm pairs |
## home manager and user-level package managers
the system module only indexes packages installed system-wide. for
home-manager or per-user nix profiles, run `inshellah index` against
those prefixes separately:
```sh
# home-manager / per-user profile
@ -167,35 +135,28 @@ inshellah index /etc/profiles/per-user/$USER
inshellah index ~/.nix-profile
```
this indexes into the default user cache (`$XDG_CACHE_HOME/inshellah`),
which the completer searches automatically. you can re-run this after
installing new packages, or add it to a home-manager activation script.
if you want to automate this in home-manager:
this indexes into `$XDG_CACHE_HOME/inshellah`, which the completer
searches automatically. to automate via home-manager:
```nix
# home.nix
home.activation.inshellah-index = lib.hm.dag.entryAfter [ "writeBoundary" ] ''
${pkgs.inshellah}/bin/inshellah index /etc/profiles/per-user/$USER 2>/dev/null || true
'';
```
the completer will then search both the system index and the user
cache, so completions from both sources are available.
## troubleshooting
**completions not appearing**: ensure the completer is configured in
your nushell config (see above). check that the system index exists:
`ls /run/current-system/sw/share/inshellah/`.
**completions not appearing**: check that the system index exists
(`ls /run/current-system/sw/share/inshellah/`) and that the completer
is configured.
**missing completions for a specific command**: check if it's a nushell
built-in (`help commands | where name == "thecommand"`). built-ins are
excluded because nushell serves its own completions for them.
built-in (`help commands | where name == "thecommand"`) — built-ins
are excluded.
**stale completions after update**: completions regenerate on every
`nixos-rebuild`. if a command changed its flags, rebuild to pick up
the changes.
**stale completions after update**: the index regenerates on every
`nixos-rebuild`. if a command changed its flags, rebuild.
**build-time errors**: indexing failures are non-fatal (`|| true`).
check `journalctl` for the build log if completions are missing.
**build-time errors**: indexing failures are non-fatal. check
`journalctl` for the build log if completions are missing for a
specific command.

View file

@ -1,150 +1,28 @@
# using inshellah completions in nushell
inshellah indexes completions from three sources (in priority order):
1. **native generators** — programs that can emit nushell completions directly
2. **manpages** — groff/troff/mdoc manpage parsing
3. **`--help` output** — parsing help text as a fallback
indexed data is stored as `.json` and `.nu` files in a directory that the
`complete` command reads from at tab-completion time.
inshellah indexes completions for the commands in your `$PATH` and
serves them to nushell's external completer. indexed data is stored as
`.json` and `.nu` files that the `complete` command reads at
tab-completion time.
## quick start
index completions from a system prefix:
```sh
# index from a prefix containing bin/ and share/man/
# from a prefix containing bin/ and share/man/
inshellah index /usr
# index from multiple prefixes
# multiple prefixes
inshellah index /usr /usr/local
# store in a custom directory
# custom directory
inshellah index /usr --dir ~/my-completions
```
parse a single manpage:
```sh
inshellah manpage /usr/share/man/man1/git.1.gz
```
batch-process all manpages under a directory (man1 and man8):
```sh
inshellah manpage-dir /usr/share/man
```
## commands
```
inshellah index PREFIX... [--dir PATH] [--ignore FILE] [--help-only FILE]
index completions into a directory of json/nu files.
PREFIX is a directory containing bin/ and share/man/.
default dir: $XDG_CACHE_HOME/inshellah
--ignore FILE skip listed commands entirely
--help-only FILE skip manpages for listed commands, use --help instead
inshellah complete CMD [ARGS...] [--dir PATH[:PATH...]]
nushell custom completer. outputs json completion candidates.
falls back to --help resolution if command is not indexed.
--dir takes colon-separated paths. the first path is the writable
user cache; additional paths are read-only system directories.
manpages are found via sibling share/man of system dir paths.
inshellah query CMD [--dir PATH[:PATH...]]
print stored completion data for CMD.
inshellah dump [--dir PATH[:PATH...]]
list indexed commands.
inshellah manpage FILE
parse a manpage and emit nushell extern block.
inshellah manpage-dir DIR
batch-process manpages under DIR (man1 and man8 sections).
```
## the index pipeline
the `index` command runs a three-phase pipeline over all executables
in each `PREFIX/bin`:
### phase 1: native completion detection (parallel)
for each executable, inshellah scans the elf binary for the string
`completion`. if found, it probes common patterns like
`CMD completions nushell` to see if the program can generate its own
nushell completions. native output is used verbatim — these are always
higher quality than parsed completions.
programs like `niri`, and any clap/cobra tool with nushell support,
are handled this way.
### phase 2: manpage parsing (sequential)
for commands not covered by phase 1, inshellah parses manpages from
man1 (user commands) and man8 (sysadmin commands). it handles:
- gnu `.TP` style (coreutils, help2man)
- `.IP` style (curl, hand-written)
- `.PP`+`.RS`/`.RE` style (git, docbook)
- nix3 bullet+hyperlink style (`nix run`, `nix build`, etc.)
- mdoc (bsd) format
- deroff fallback for unusual formats
synopsis sections are parsed to detect subcommands: `git-commit.1`
generates `export extern "git commit"`, not `export extern "git-commit"`.
### phase 3: --help fallback (parallel)
remaining executables without manpages get `--help` (or `-h`) called
with a 200ms timeout. elf binaries are pre-scanned for the `-h` string
to skip those that don't support help flags. shell scripts are run
directly (they're fast). execution is parallelized to available cores.
subcommands are recursively resolved — if `--help` output lists
subcommands, inshellah runs `CMD SUBCMD --help` for each.
when a `--help` invocation produces rendered manpage output (some
commands like `git stash` delegate `--help` to `man`), inshellah
detects this and locates the raw manpage source to parse with the
groff parser instead. this yields richer results (subcommands,
structured flag sections) than parsing the rendered text.
### output
each command gets its own file in the index directory. native generators
produce `.nu` files; parsed results produce `.json` files. the `complete`
command reads both formats.
nushell built-in commands (ls, cd, cp, mv, etc.) are excluded since
nushell provides its own completions.
### performance
on a typical nixos system (~950 executables, ~1600 manpages):
- total time: ~4-10 seconds
- native gzip decompression (camlzip, no process spawning)
- parallel --help with core-scaled forking
- elf string scanning to skip ~15% of binaries
## the completer
the `complete` command is designed to be wired into nushell as an
external completer. it reads from the directories specified via `--dir`
(colon-separated), performs fuzzy matching, and outputs json completion
candidates. the first path is the writable user cache; additional paths
are read-only system directories.
if a command is not indexed, `complete` falls back to on-the-fly
`--help` resolution — it runs the command's help, caches the result
in the user directory, and returns completions immediately.
### setting up the completer
then wire up the completer in `~/.config/nushell/config.nu`:
```nu
# ~/.config/nushell/config.nu
$env.config.completions.external = {
enable: true
completer: {|spans|
@ -154,27 +32,62 @@ $env.config.completions.external = {
}
```
with the nixos module, use the provided `snippet` option value (see
[nixos.md](nixos.md)) which points at the system index automatically.
that's it. tab-completion now works for every command indexed.
## nixos module
## commands
enable automatic completion indexing at system build time:
```
inshellah index PREFIX... [--dir PATH] [--ignore FILE] [--help-only FILE]
[--workers N] [--timeout-ms N]
index completions into a directory of json/nu files.
PREFIX is a directory containing bin/ and share/man/.
default dir: $XDG_CACHE_HOME/inshellah
--ignore FILE skip listed commands entirely
--help-only FILE skip manpages for listed commands, use --help instead
--workers N worker-thread count
--timeout-ms N per-subprocess timeout in ms (default: 200)
```nix
{
imports = [ ./path/to/inshellah/nix/module.nix ];
programs.inshellah.enable = true;
}
inshellah complete CMD [ARGS...] [--dir PATH[:PATH...]] [--timeout-ms N]
nushell custom completer. outputs JSON completion candidates.
falls back to on-the-fly --help resolution if a command isn't
indexed yet — the result is cached and subsequent presses are
instant.
--dir takes colon-separated paths. the first path is the writable
user cache; additional paths are read-only system directories.
inshellah query CMD [--dir PATH[:PATH...]]
print stored completion data for CMD.
inshellah dump [--dir PATH[:PATH...]]
list indexed commands.
inshellah manpage FILE
parse a manpage and emit a nushell extern block.
inshellah manpage-dir DIR
batch-process manpages under DIR (man1 and man8 sections).
```
this runs `inshellah index` during the system profile build. see
[nixos.md](nixos.md) for full details.
## what gets handled
## what gets generated
- **sources**: native nushell completion generators (clap/cobra tools
that can emit completions themselves), manpages in section 1 and 8,
`--help` and `-h` output.
- **groff styles**: gnu `.TP` (coreutils, help2man), `.IP` (curl,
hand-written), `.PP`+`.RS`/`.RE` (git, docbook), nix3 bullet
(`nix run`, `nix build`), mdoc (BSD), plus a deroff fallback.
- **subcommand naming**: `git-commit.1` produces `git commit`, not
`git-commit`. clap-style per-subcommand manpages get one file each.
- **synopsis-only flags**: flags declared in a manpage SYNOPSIS but
missing from the body (e.g. nix-env's `--profile`, most of sed's
interface) are picked up too.
- **elevation wrappers**: `sudo`, `doas`, `pkexec`, `su`, `run0` are
stripped before lookup, including when the real target is given as
an absolute path.
- **exclusions**: nushell built-ins (ls, cd, mv, etc.) are skipped —
nushell serves its own completions for those.
the `manpage` and `manpage-dir` commands emit nushell `extern` blocks
with flags, parameter types, and descriptions:
## extern blocks (manpage / manpage-dir)
```nu
export extern "rg" [
@ -186,9 +99,12 @@ export extern "rg" [
]
```
subcommand manpages (e.g. `git-commit.1`) are detected via synopsis
parsing and generate the correct nushell name (`git commit` not
`git-commit`).
these are produced by `inshellah manpage` / `inshellah manpage-dir` and
can be source'd directly in your nushell config if you prefer that to
the json completer flow.
nushell built-in commands (ls, cd, mv, etc.) are excluded since nushell
provides its own completions for these.
## nixos
`programs.inshellah.enable = true` will index at system build time and
ship a richer completer with runtime fallbacks (live cluster queries,
git/ssh/docker/k8s lookups, etc.). see [nixos.md](nixos.md).

View file

@ -1,30 +1,31 @@
# runtime completion resolution
the `complete` command has built-in on-the-fly resolution: when a command
is not found in the index, it falls back to running `--help`, caches the
result, and returns completions immediately. this means commands installed
outside the system profile (via cargo, pip, npm, go, etc.) get completions
on first tab-press with no manual setup.
when a command isn't in the static index yet, `inshellah complete`
runs `--help` (or `-h`) on the binary, caches the result in the user
directory, and returns completions immediately. tab-completion just
works for tools installed outside the indexed prefixes — via cargo,
pip, npm, go, etc.
## how it works
when you type `docker compose up --<TAB>`:
typing `docker compose up --<TAB>`:
1. nushell calls `inshellah complete docker compose up --`
2. inshellah looks up the index for the longest matching prefix
2. inshellah looks up the longest matching prefix in the index
3. if found, it fuzzy-matches flags and subcommands against the partial input
4. if not found, it locates the binary in `$PATH`, runs `--help`,
recursively resolves subcommands, caches the results in the user
directory (`$XDG_CACHE_HOME/inshellah`), and returns completions.
if `--help` produces rendered manpage output, the raw manpage source
is located and parsed instead for richer results
directory (`$XDG_CACHE_HOME/inshellah`), and returns completions
all subsequent completions for that command are instant (served from cache).
all subsequent completions for that command are served from cache.
elevation wrappers (`sudo`, `doas`, `pkexec`, `su`, `run0`) are
stripped before lookup: `sudo docker compose up --` resolves against
`docker`, not `sudo`. absolute paths after the wrapper are recognised
too.
## setup
the completer works with no extra configuration beyond the basic setup:
```nu
# ~/.config/nushell/config.nu
$env.config.completions.external = {
@ -36,18 +37,8 @@ $env.config.completions.external = {
}
```
with the nixos module, the installed wrapper has the system paths
hardcoded — no extra flags needed. the same snippet works:
```nu
$env.config.completions.external = {
enable: true
completer: {|spans|
inshellah complete ...$spans
| from json
}
}
```
with the nixos module, no extra config is needed beyond enabling the
module — the wrapper has the system paths baked in.
to manually specify system dirs, use colon-separated `--dir`:
@ -61,25 +52,15 @@ $env.config.completions.external = {
}
```
system directories (paths after the first in `--dir`) enable
manpage-based fallback: when a command's `--help` delegates to `man`,
the completer looks for the raw manpage in the sibling `share/man`
directory (e.g. `share/inshellah``share/man`). if no system dirs
are given, it falls back to `man -w` to locate the manpage.
or use the `snippet` option provided by the flake module (see
[nixos.md](nixos.md)).
paths after the first in `--dir` are read-only system dirs.
## cache management
the user cache lives at `$XDG_CACHE_HOME/inshellah` (typically
`~/.cache/inshellah`).
```sh
# list cached commands
inshellah dump
# view cached data for a command
# view stored data for a command
inshellah query docker
# clear cache