atagen/inshellah
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

init

Browse source
This commit is contained in:
atagen 2026-03-18 15:40:47 +11:00
commit bef1adc62a
23 changed files with 5277 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

163
doc/building.md Normal file
View file

@ -0,0 +1,163 @@
# building and installing
## dependencies
inshellah is written in OCaml and uses dune as its build system.
build dependencies:
- **OCaml** >= 5.0
- **dune** >= 3.20
- **angstrom** — parser combinator library
- **angstrom-unix** — unix extensions for angstrom
- **camlzip** — gzip decompression for reading compressed manpages
- **str** — regular expressions (ships with OCaml)
- **unix** — process/file operations (ships with OCaml)
runtime dependencies:
- **man** (optional) — used as a fallback to locate manpages during
on-the-fly completion resolution. not needed if `--system-dir` is
provided (manpages are found via sibling `share/man`).
## building with nix (recommended)
if you have nix installed:
```sh
nix build
```
the binary is at `./result/bin/inshellah`.
for development with a shell containing all dependencies:
```sh
nix develop
dune build
dune test
```
## building from source with opam
install dependencies via opam:
```sh
opam install dune angstrom angstrom-unix camlzip
```
build and test:
```sh
dune build
dune test
```
install into the opam switch:
```sh
dune install
```
## building from source without opam
if your distribution packages the OCaml libraries directly, install
them through your package manager, then build with dune:
```sh
dune build
```
the binary is at `_build/default/bin/main.exe`. copy it to your
`$PATH`:
```sh
install -Dm755 _build/default/bin/main.exe /usr/local/bin/inshellah
```
## arch linux
install OCaml and dune from the official repos, and the remaining
libraries from the AUR or via opam:
```sh
# system packages
sudo pacman -S ocaml dune
# ocaml libraries (via opam)
opam init # if not already initialized
eval $(opam env)
opam install angstrom angstrom-unix camlzip
# build
dune build
dune test
# install
sudo install -Dm755 _build/default/bin/main.exe /usr/local/bin/inshellah
```
## debian / ubuntu
```sh
sudo apt install ocaml opam
opam init
eval $(opam env)
opam install dune angstrom angstrom-unix camlzip
dune build
sudo install -Dm755 _build/default/bin/main.exe /usr/local/bin/inshellah
```
## fedora
```sh
sudo dnf install ocaml opam
opam init
eval $(opam env)
opam install dune angstrom angstrom-unix camlzip
dune build
sudo install -Dm755 _build/default/bin/main.exe /usr/local/bin/inshellah
```
## post-install setup
after installing the binary, index completions from your system
prefix(es):
```sh
# typical linux system
inshellah index /usr /usr/local
# check what was indexed
inshellah dump
```
then wire up the nushell completer:
```nu
# ~/.config/nushell/config.nu
$env.config.completions.external = {
enable: true
completer: {|spans|
inshellah complete ...$spans
| from json
}
}
```
see [nushell-integration.md](nushell-integration.md) for full details
on the completer, and [runtime-completions.md](runtime-completions.md)
for on-the-fly resolution of commands not covered by the index.
## re-indexing after package changes
the index is a static cache — it doesn't update automatically when you
install or remove packages. re-run `inshellah index` after significant
package changes:
```sh
inshellah index /usr /usr/local
```
on nixos, the system index regenerates on every `nixos-rebuild`
automatically. see [nixos.md](nixos.md) for details.

196
doc/nixos.md Normal file
View file

@ -0,0 +1,196 @@
# nixos integration
inshellah provides a nixos module that automatically indexes nushell
completions for all installed packages at system build time.
## enabling
```nix
# in your flake.nix outputs:
{
nixosConfigurations.myhost = nixpkgs.lib.nixosSystem {
modules = [
inshellah.nixosModules.default
{
programs.inshellah.enable = true;
}
];
};
}
```
or if importing the module directly:
```nix
# configuration.nix
{ pkgs, ... }: {
imports = [ ./path/to/inshellah/nix/module.nix ];
programs.inshellah = {
enable = true;
package = pkgs.inshellah; # or your local build
};
}
```
## what happens at build time
the module hooks into `environment.extraSetup`, which runs during the
system profile build (the `buildEnv` that creates `/run/current-system/sw`).
at that point, all system packages are merged, so `$out/bin` contains every
executable and `$out/share/man` contains every manpage.
inshellah runs a single command:
```
inshellah index "$out" --dir $out/share/inshellah
```
this executes a three-phase pipeline:
### phase 1: native completion detection (parallel)
for each executable, inshellah scans the elf binary for the string
`completion`. if found, it probes common patterns like
`CMD completions nushell` to see if the program can generate its own
nushell completions. native output is used verbatim — these are always
higher quality than parsed completions.
programs like `niri`, and any clap/cobra tool with nushell support,
are handled this way.
### phase 2: manpage parsing (sequential)
for commands not covered by phase 1, inshellah parses manpages from
man1 (user commands) and man8 (sysadmin commands). it handles:
- gnu `.TP` style (coreutils, help2man)
- `.IP` style (curl, hand-written)
- `.PP`+`.RS`/`.RE` style (git, docbook)
- nix3 bullet+hyperlink style (`nix run`, `nix build`, etc.)
- mdoc (bsd) format
- deroff fallback for unusual formats
synopsis sections are parsed to detect subcommands: `git-commit.1`
generates `export extern "git commit"`, not `export extern "git-commit"`.
### phase 3: --help fallback (parallel)
remaining executables without manpages get `--help` (or `-h`) called
with a 200ms timeout. elf binaries are pre-scanned for the `-h` string
to skip those that don't support help flags. shell scripts are run
directly (they're fast). execution is parallelized to available cores.
when `--help` produces rendered manpage output instead of plain help
text (e.g. `git stash --help` delegates to `man`), the raw manpage
source is located and parsed with the groff parser for richer results.
### output
each command gets its own file in `/share/inshellah` under the system
profile. native generators produce `.nu` files; parsed results produce
`.json` files. the `complete` command reads both formats.
nushell built-in commands (ls, cd, cp, mv, etc.) are excluded since
nushell provides its own completions.
### performance
on a typical nixos system (~950 executables, ~1600 manpages):
- total time: ~4-10 seconds
- native gzip decompression (camlzip, no process spawning)
- parallel --help with core-scaled forking
- elf string scanning to skip ~15% of binaries
## module options
```nix
programs.inshellah = {
enable = true;
# the inshellah package (set automatically by the flake module)
package = pkgs.inshellah;
# where to place indexed completion files under the system profile
# default: "/share/inshellah"
completionsPath = "/share/inshellah";
# commands to skip entirely during indexing
ignoreCommands = [ "problematic-tool" ];
# commands to skip manpage parsing for (uses --help instead)
helpOnlyCommands = [ "nix" ];
};
```
## using the completer
the flake module sets a read-only `snippet` option containing the nushell
config needed to wire up the completer. you can access it via
`config.programs.inshellah.snippet` and paste it into your nushell config,
or source it from a file generated by your nixos config.
the snippet sets up the external completer pointing at the system index
at `/run/current-system/sw/share/inshellah`:
```nu
let inshellah_complete = {|spans|
inshellah complete ...$spans --system-dir /run/current-system/sw/share/inshellah | from json
}
$env.config.completions.external = {
enable: true
max_results: 100
completer: $inshellah_complete
}
```
## home manager and other user-level package managers
the nixos module only indexes packages installed at the system level
(those that end up in `/run/current-system/sw`). if you use home-manager,
nix-env, or another user-level package manager, those binaries and
manpages live elsewhere — typically under `/etc/profiles/per-user/<name>`
or `~/.nix-profile`.
to get completions for user-installed packages, run `inshellah index`
against those prefixes separately:
```sh
# home-manager / per-user profile
inshellah index /etc/profiles/per-user/$USER
# classic nix-env profile
inshellah index ~/.nix-profile
```
this indexes into the default user cache (`$XDG_CACHE_HOME/inshellah`),
which the completer searches automatically. you can re-run this after
installing new packages, or add it to a home-manager activation script.
if you want to automate this in home-manager:
```nix
# home.nix
home.activation.inshellah-index = lib.hm.dag.entryAfter [ "writeBoundary" ] ''
${pkgs.inshellah}/bin/inshellah index /etc/profiles/per-user/$USER 2>/dev/null || true
'';
```
the completer will then search both the system index (`--system-dir`)
and the user cache, so completions from both sources are available.
## troubleshooting
**completions not appearing**: ensure the completer is configured in
your nushell config (see above). check that the system index exists:
`ls /run/current-system/sw/share/inshellah/`.
**missing completions for a specific command**: check if it's a nushell
built-in (`help commands | where name == "thecommand"`). built-ins are
excluded because nushell serves its own completions for them.
**stale completions after update**: completions regenerate on every
`nixos-rebuild`. if a command changed its flags, rebuild to pick up
the changes.
**build-time errors**: indexing failures are non-fatal (`|| true`).
check `journalctl` for the build log if completions are missing.

191
doc/nushell-integration.md Normal file
View file

@ -0,0 +1,191 @@
# using inshellah completions in nushell
inshellah indexes completions from three sources (in priority order):
1. **native generators** — programs that can emit nushell completions directly
2. **manpages** — groff/troff/mdoc manpage parsing
3. **`--help` output** — parsing help text as a fallback
indexed data is stored as `.json` and `.nu` files in a directory that the
`complete` command reads from at tab-completion time.
## quick start
index completions from a system prefix:
```sh
# index from a prefix containing bin/ and share/man/
inshellah index /usr
# index from multiple prefixes
inshellah index /usr /usr/local
# store in a custom directory
inshellah index /usr --dir ~/my-completions
```
parse a single manpage:
```sh
inshellah manpage /usr/share/man/man1/git.1.gz
```
batch-process all manpages under a directory (man1 and man8):
```sh
inshellah manpage-dir /usr/share/man
```
## commands
```
inshellah index PREFIX... [--dir PATH] [--ignore FILE] [--help-only FILE]
index completions into a directory of json/nu files.
PREFIX is a directory containing bin/ and share/man/.
default dir: $XDG_CACHE_HOME/inshellah
--ignore FILE skip listed commands entirely
--help-only FILE skip manpages for listed commands, use --help instead
inshellah complete CMD [ARGS...] [--dir PATH] [--system-dir PATH]
nushell custom completer. outputs json completion candidates.
falls back to --help resolution if command is not indexed.
manpages are found via sibling share/man of --system-dir paths.
inshellah query CMD [--dir PATH] [--system-dir PATH]
print stored completion data for CMD.
inshellah dump [--dir PATH] [--system-dir PATH]
list indexed commands.
inshellah manpage FILE
parse a manpage and emit nushell extern block.
inshellah manpage-dir DIR
batch-process manpages under DIR (man1 and man8 sections).
```
## the index pipeline
the `index` command runs a three-phase pipeline over all executables
in each `PREFIX/bin`:
### phase 1: native completion detection (parallel)
for each executable, inshellah scans the elf binary for the string
`completion`. if found, it probes common patterns like
`CMD completions nushell` to see if the program can generate its own
nushell completions. native output is used verbatim — these are always
higher quality than parsed completions.
programs like `niri`, and any clap/cobra tool with nushell support,
are handled this way.
### phase 2: manpage parsing (sequential)
for commands not covered by phase 1, inshellah parses manpages from
man1 (user commands) and man8 (sysadmin commands). it handles:
- gnu `.TP` style (coreutils, help2man)
- `.IP` style (curl, hand-written)
- `.PP`+`.RS`/`.RE` style (git, docbook)
- nix3 bullet+hyperlink style (`nix run`, `nix build`, etc.)
- mdoc (bsd) format
- deroff fallback for unusual formats
synopsis sections are parsed to detect subcommands: `git-commit.1`
generates `export extern "git commit"`, not `export extern "git-commit"`.
### phase 3: --help fallback (parallel)
remaining executables without manpages get `--help` (or `-h`) called
with a 200ms timeout. elf binaries are pre-scanned for the `-h` string
to skip those that don't support help flags. shell scripts are run
directly (they're fast). execution is parallelized to available cores.
subcommands are recursively resolved — if `--help` output lists
subcommands, inshellah runs `CMD SUBCMD --help` for each.
when a `--help` invocation produces rendered manpage output (some
commands like `git stash` delegate `--help` to `man`), inshellah
detects this and locates the raw manpage source to parse with the
groff parser instead. this yields richer results (subcommands,
structured flag sections) than parsing the rendered text.
### output
each command gets its own file in the index directory. native generators
produce `.nu` files; parsed results produce `.json` files. the `complete`
command reads both formats.
nushell built-in commands (ls, cd, cp, mv, etc.) are excluded since
nushell provides its own completions.
### performance
on a typical nixos system (~950 executables, ~1600 manpages):
- total time: ~4-10 seconds
- native gzip decompression (camlzip, no process spawning)
- parallel --help with core-scaled forking
- elf string scanning to skip ~15% of binaries
## the completer
the `complete` command is designed to be wired into nushell as an
external completer. it reads from the index directory (`--dir`) and
optional system directories (`--system-dir`), performs fuzzy matching,
and outputs json completion candidates.
if a command is not indexed, `complete` falls back to on-the-fly
`--help` resolution — it runs the command's help, caches the result
in the user directory, and returns completions immediately.
### setting up the completer
```nu
# ~/.config/nushell/config.nu
$env.config.completions.external = {
enable: true
completer: {|spans|
inshellah complete ...$spans
| from json
}
}
```
with the nixos module, use the provided `snippet` option value (see
[nixos.md](nixos.md)) which points at the system index automatically.
## nixos module
enable automatic completion indexing at system build time:
```nix
{
imports = [ ./path/to/inshellah/nix/module.nix ];
programs.inshellah.enable = true;
}
```
this runs `inshellah index` during the system profile build. see
[nixos.md](nixos.md) for full details.
## what gets generated
the `manpage` and `manpage-dir` commands emit nushell `extern` blocks
with flags, parameter types, and descriptions:
```nu
export extern "rg" [
--regexp(-e): string # a pattern to search for
--file(-f): path # search for patterns from the given file
--count(-c) # only show the count of matching lines
--color: string # controls when to use color
--max-depth: int # limit the depth of directory traversal
]
```
subcommand manpages (e.g. `git-commit.1`) are detected via synopsis
parsing and generate the correct nushell name (`git commit` not
`git-commit`).
nushell built-in commands (ls, cd, mv, etc.) are excluded since nushell
provides its own completions for these.

92
doc/runtime-completions.md Normal file
View file

@ -0,0 +1,92 @@
# runtime completion resolution
the `complete` command has built-in on-the-fly resolution: when a command
is not found in the index, it falls back to running `--help`, caches the
result, and returns completions immediately. this means commands installed
outside the system profile (via cargo, pip, npm, go, etc.) get completions
on first tab-press with no manual setup.
## how it works
when you type `docker compose up --<TAB>`:
1. nushell calls `inshellah complete docker compose up --`
2. inshellah looks up the index for the longest matching prefix
3. if found, it fuzzy-matches flags and subcommands against the partial input
4. if not found, it locates the binary in `$PATH`, runs `--help`,
recursively resolves subcommands, caches the results in the user
directory (`$XDG_CACHE_HOME/inshellah`), and returns completions.
if `--help` produces rendered manpage output, the raw manpage source
is located and parsed instead for richer results
all subsequent completions for that command are instant (served from cache).
## setup
the completer works with no extra configuration beyond the basic setup:
```nu
# ~/.config/nushell/config.nu
$env.config.completions.external = {
enable: true
completer: {|spans|
inshellah complete ...$spans
| from json
}
}
```
with the nixos module, add `--system-dir` to also search the system index:
```nu
$env.config.completions.external = {
enable: true
completer: {|spans|
inshellah complete ...$spans --system-dir /run/current-system/sw/share/inshellah
| from json
}
}
```
`--system-dir` also enables manpage-based fallback: when a command's
`--help` delegates to `man`, the completer looks for the raw manpage
in the sibling `share/man` directory (e.g. `share/inshellah` →
`share/man`). if no `--system-dir` is given, it falls back to
`man -w` to locate the manpage.
or use the `snippet` option provided by the flake module (see
[nixos.md](nixos.md)).
## cache management
the user cache lives at `$XDG_CACHE_HOME/inshellah` (typically
`~/.cache/inshellah`).
```sh
# list cached commands
inshellah dump
# view cached data for a command
inshellah query docker
# clear cache
rm -rf ~/.cache/inshellah/
# re-index from a prefix
inshellah index /usr --dir ~/.cache/inshellah
```
## when to use this vs build-time indexing
the nixos module (`programs.inshellah.enable = true`) handles system
packages at build time. runtime resolution covers:
- commands installed outside the system profile (cargo, pip, npm, go)
- subcommand completions at arbitrary depth
- systems without the nixos module
for upfront indexing on non-nixos systems:
```sh
inshellah index /usr /usr/local
```