Phase 2: view-only inventory and duplicate detection across package managers

README.md

@@ -14,21 +14,32 @@ A fast, cross-platform Terminal User Interface (TUI) for browsing the packages i
 
 pkgmate is a viewer today. It does:
 
-- List installed packages from every supported source that is present on the machine, in four tabs: direct, dependency, all and orphan packages.
+- List installed packages from every supported source that is present on the machine, in five tabs: direct, dependency, all and orphan packages, plus duplicates.
 - Mark packages that are frozen (held or pinned) and packages with a newer version available. For pacman, the newer version of AUR packages is looked up from the AUR.
 - Search the list with live fuzzy matching.
-- Refresh package databases ("update") for pacman, apt and Homebrew, running the privileged commands as described in [Privileged commands](#privileged-commands). The update flow is implemented in the backends but is not yet listed in the command palette.
 
-pkgmate does not yet install, remove or upgrade packages. Flatpak, Snap, npm, pip and Go have no update support either: they are skipped when an update runs.
+pkgmate does not yet install, remove or upgrade packages.
+
+### Duplicates
+
+- A **duplicate** is the same application installed through two or more different package managers, with at least one of the copies installed explicitly. pacman and its AUR packages count as one manager, and so do brew formulae and casks.
+- The **Duplicates** tab lists the copies of the same app on adjacent rows, the group with the largest combined size first. In the other tabs, the **Package Manager** cell of a grouped package names the other managers in its group, for example `pacman (+flatpak)`, and the footer reports how many apps are installed more than once.
+- Names are matched with a small set of per-ecosystem rules:
+  - names are compared lowercase, with `_` treated as `-`;
+  - Python packages match across pip, pipx, pacman `python-*` and apt `python3-*`;
+  - a trailing `-bin`, `-git` or `-appimage` is ignored on pacman names;
+  - Go binaries match by their file name (the command you run, e.g. `gopls`), not by the module they were built from;
+  - Flatpak apps match by the last part of their application ID, with a small built-in alias list for well-known apps whose ID does not normalise to their common name;
+  - **npm global packages are matched by name alone, so an unrelated package that happens to share a name can be reported.**
+- pkgmate only reports duplicates. It never removes anything.
 
 ## Features
 
 - ⚡ **Fast startup** - Direct file parsing and efficient CLI wrapping
 - 🔍 **Telescope-style search** - Live fuzzy search through packages
 - 📦 **Multi-backend support** - pacman/AUR, dpkg/apt, Homebrew, Flatpak, Snap, npm, pip/pipx, Go
 - 🔒 **Security-first** - GPG signing, automated vulnerability scanning
-- 📊 **Clean interface** - Table view with virtual scrolling, visual indicators for available updates and frozen packages, plus an orphan tab
-- ⚡ **Privilege escalation** - pkexec (PolicyKit) with sudo fallback
+- 📊 **Clean interface** - Table view with virtual scrolling, visual indicators for available updates and frozen packages, plus orphan and duplicates tabs
 - 🎨 **Command palette** - Quick access to all features (Ctrl+k or Ctrl+p)
 
 ## Supported Sources
@@ -143,23 +154,24 @@ pkgmate --version
 - `Ctrl+P` or `Ctrl+k` - Command palette
 - `Ctrl+C` - Quit
 
-## Privileged commands
-
-Refreshing a package database needs root, so pkgmate itself refuses to start as root and escalates only for the command that needs it. For each privileged command it:
+## Security model
 
-1. Resolves the command to an absolute path.
-2. Runs it through `pkexec <path> <args>`. If `pkexec` is not installed, cannot be started, or exits with 126 or 127 (authentication dismissed or not authorized), it retries with `sudo -- <path> <args>`.
-3. Runs the argv directly, without a shell, so no argument is ever interpreted by one.
+pkgmate only reads. It refuses to start as root, it never calls `sudo` or `pkexec`, and every external command is run as argv with no shell.
 
-The exact argv is written to the application log (Debug Console). The command's output is shown live in the terminal while it runs, and kept so pkgmate can show it again afterwards.
+These are the queries it runs, all as your user. The Network column says whether the query uses the network.
 
-The commands pkgmate runs today:
+| Source | Queries | Network |
+|--------|---------|---------|
+| pacman | `pacman -Qi`, `-Qeq`, `-Qmq`, `-Qu`, `-Qtdq`; reads `/etc/pacman.conf`; `checkupdates --nocolor` when installed (pacman-contrib) | `checkupdates` syncs a private copy of the sync databases under `$TMPDIR/checkup-db-$UID` with `fakeroot`; the system databases are not touched. The AUR RPC (`https://aur.archlinux.org/rpc/v5/info`) is an HTTPS lookup for AUR versions. |
+| dpkg/apt | `dpkg --get-selections`, `apt list --upgradable`, `apt-get --simulate autoremove`; reads `/var/lib/dpkg/status` | none |
+| Homebrew | `brew --cellar`, `brew --prefix`, `brew outdated --json=v2`, `brew autoremove --dry-run` | |
+| Flatpak | `flatpak list --app [--columns=…]` | |
+| Snap | `snap list`, `snap refresh --list` | `snap refresh --list` asks snapd and the store; it does not refresh |
+| npm | `npm root -g`, `npm list -g --json --depth=0` | |
+| pip | `pip`/`pip3 list --format=json`, `pip show pip`, `pip list --outdated --format=json`, `pipx list [--json]` | `pip list --outdated` queries the package index |
+| Go | reads build info of binaries in `$GOBIN`, `$GOPATH/bin` or `~/go/bin`; no command | |
 
-| Source | Command | Escalation |
-|--------|---------|------------|
-| pacman | `pacman -Syy --noconfirm` | pkexec, then sudo |
-| dpkg/apt | `apt-get update`, or `apt update` when `apt-get` is missing | pkexec, then sudo |
-| Homebrew | `brew update` | none, runs as your user |
+The crash screen can also pipe the report to `pbcopy`, `wl-copy`, `xclip` or `xsel` when you ask it to.
 
 ## Verifying Downloads
 

backend/backend.go

@@ -1,21 +1,16 @@
 package backend
 
 import (
-	"errors"
 	"fmt"
-	"io"
 	"io/fs"
 	"log/slog"
-	"os"
 	"path/filepath"
-	"strings"
 	"sync"
 	"time"
 )
 
-var ErrUpdateNotImplemented = errors.New("update not implemented")
-
 type Package struct {
+	ID           string // native identifier when it differs from Name (Flatpak application ID, Go binary file name); empty otherwise
 	Name         string
 	Version      string
 	NewVersion   string
@@ -51,7 +46,6 @@ type Backend interface {
 	Name() string
 	IsAvailable() bool
 	LoadPackages(*sync.WaitGroup, func(Package))
-	Update() (func() error, chan OperationResult)
 }
 
 var (
@@ -106,118 +100,6 @@ func LoadAllPackages() (chan Package, error) {
 	return outChan, nil
 }
 
-func UpdateAll() (func() error, chan OperationResult) {
-	return updateBackends(GetAvailableBackends(), os.Stdout)
-}
-
-func updateBackends(backends []Backend, progress io.Writer) (func() error, chan OperationResult) {
-	if progress == nil {
-		progress = io.Discard
-	}
-	resultChan := make(chan OperationResult, 1)
-
-	execFunc := func() error {
-		var updated []string
-		var skipped []string
-		type failedBackend struct {
-			name string
-			err  error
-		}
-		var failed []failedBackend
-		type backendLog struct {
-			name string
-			logs string
-		}
-		var logs []backendLog
-
-		for _, b := range backends {
-			name := b.Name()
-			fmt.Fprintf(progress, "pkgmate: updating %s…\n", name)
-
-			f, ch := b.Update()
-			if f == nil {
-				skipped = append(skipped, name)
-				fmt.Fprintf(progress, "pkgmate: %s skipped\n", name)
-				continue
-			}
-
-			err := f()
-			if err != nil {
-				if errors.Is(err, ErrUpdateNotImplemented) {
-					skipped = append(skipped, name)
-					fmt.Fprintf(progress, "pkgmate: %s skipped\n", name)
-					continue
-				}
-				failed = append(failed, failedBackend{name: name, err: err})
-				fmt.Fprintf(progress, "pkgmate: %s failed\n", name)
-				continue
-			}
-
-			res := <-ch
-			if res.Logs != "" {
-				logs = append(logs, backendLog{name: name, logs: res.Logs})
-			}
-
-			if res.Success {
-				updated = append(updated, name)
-				fmt.Fprintf(progress, "pkgmate: %s updated\n", name)
-			} else {
-				resErr := res.Error
-				if resErr == nil {
-					resErr = errors.New("update failed")
-				}
-				failed = append(failed, failedBackend{name: name, err: resErr})
-				fmt.Fprintf(progress, "pkgmate: %s failed\n", name)
-			}
-		}
-
-		var finalErr error
-		if len(failed) > 0 {
-			errs := make([]error, len(failed))
-			for i, f := range failed {
-				errs[i] = fmt.Errorf("%s: %w", f.name, f.err)
-			}
-			finalErr = errors.Join(errs...)
-		}
-
-		var sb strings.Builder
-		if len(backends) == 0 {
-			sb.WriteString("No available backends\n")
-		} else {
-			if len(updated) > 0 {
-				sb.WriteString("Updated: " + strings.Join(updated, ", ") + "\n")
-			}
-			if len(skipped) > 0 {
-				sb.WriteString("Skipped: " + strings.Join(skipped, ", ") + "\n")
-			}
-			if len(failed) > 0 {
-				failedStrs := make([]string, len(failed))
-				for i, f := range failed {
-					failedStrs[i] = fmt.Sprintf("%s: %v", f.name, f.err)
-				}
-				sb.WriteString("Failed: " + strings.Join(failedStrs, ", ") + "\n")
-			}
-			for _, l := range logs {
-				sb.WriteString("\n==> " + l.name + "\n")
-				sb.WriteString(l.logs)
-				if !strings.HasSuffix(l.logs, "\n") {
-					sb.WriteString("\n")
-				}
-			}
-		}
-
-		resultChan <- OperationResult{
-			Success: len(failed) == 0,
-			Error:   finalErr,
-			Logs:    sb.String(),
-		}
-
-		return nil
-	}
-
-	return execFunc, resultChan
-}
-
 func calculateSize(path string) int64 {
 	var size int64
 	err := filepath.WalkDir(path, func(_ string, d fs.DirEntry, err error) error {

backend/backend_test.go

@@ -30,10 +30,6 @@ func (f *fakeNestedBackend) LoadPackages(wg *sync.WaitGroup, send func(Package))
 	})
 }
 
-func (f *fakeNestedBackend) Update() (func() error, chan OperationResult) {
-	return nil, nil
-}
-
 func TestLoadAllPackages(t *testing.T) {
 	backendMutex.Lock()
 	origBackends := registeredBackends

backend/brew.go

@@ -179,10 +179,6 @@ func (b *BrewBackend) LoadPackages(wg *sync.WaitGroup, load func(Package)) {
 	})
 }
 
-func (b *BrewBackend) Update() (func() error, chan OperationResult) {
-	return createNormalCmd("update", "brew", "update")
-}
-
 func parseInstallReceipt(data []byte) (installDate time.Time, isDirect bool) {
 	var receipt installReceipt
 	isDirect = true

backend/dpkg.go

@@ -74,24 +74,6 @@ func (d *DpkgBackend) LoadPackages(wg *sync.WaitGroup, load func(Package)) {
 	})
 }
 
-func (d *DpkgBackend) Update() (func() error, chan OperationResult) {
-	if _, err := exec.LookPath("apt-get"); err == nil {
-		return CreatePrivilegedCmd(d.Name()+"-update", "apt-get", "update")
-	}
-
-	if _, err := exec.LookPath("apt"); err == nil {
-		return CreatePrivilegedCmd(d.Name()+"-update", "apt", "update")
-	}
-
-	resultChan := make(chan OperationResult, 1)
-	err := fmt.Errorf("neither apt-get nor apt found")
-
-	return func() error {
-		resultChan <- OperationResult{Error: err}
-		return err
-	}, resultChan
-}
-
 func parseDpkgStatus(data string) []Package {
 	var pkgs []Package
 

backend/dpkg_parse_test.go

@@ -26,6 +26,7 @@ func TestParseDpkgStatus(t *testing.T) {
 		{Name: "ca-certificates", Version: "20240203~22.04.1", Size: 0},
 		{Name: "libc6", Version: "2.35-0ubuntu3.8", Size: 13182 * 1024},
 		{Name: "libc6", Version: "2.35-0ubuntu3.8", Size: 12000 * 1024},
+		{Name: "python3-requests", Version: "2.27.1-1", Size: 450 * 1024},
 	}
 
 	got := parseDpkgStatus(readDpkgFixture(t, "status"))

backend/dummy.go

@@ -25,9 +25,6 @@ func (DummyBackend) LoadPackages() (chan []Package, error) {
 func (DummyBackend) GetOrphanPackages() ([]Package, error) {
 	return nil, nil
 }
-func (DummyBackend) Update() (func() error, chan OperationResult) {
-	return nil, nil
-}
 func (DummyBackend) String() string {
 	return "Dummy"
 }

backend/flatpak.go

@@ -44,23 +44,13 @@ func (f *FlatpakBackend) LoadPackages(wg *sync.WaitGroup, load func(Package)) {
 		}
 
 		for _, pkg := range parseFlatpakList(string(output)) {
-			pkg.Date = getFlatpakInstallDate(pkg.Name)
+			pkg.Date = getFlatpakInstallDate(pkg.ID)
 
 			load(pkg)
 		}
 	})
 }
 
-func (f *FlatpakBackend) Update() (func() error, chan OperationResult) {
-	resultChan := make(chan OperationResult, 1)
-	err := fmt.Errorf("flatpak %w", ErrUpdateNotImplemented)
-
-	return func() error {
-		resultChan <- OperationResult{Error: err}
-		return err
-	}, resultChan
-}
-
 func parseFlatpakList(out string) []Package {
 	var pkgs []Package
 
@@ -89,6 +79,7 @@ func parseFlatpakList(out string) []Package {
 			continue
 		}
 
+		appID := fields[0]
 		version := fields[sizeIdx-1]
 		sizeStr := fields[sizeIdx] + fields[sizeIdx+1]
 
@@ -98,6 +89,7 @@ func parseFlatpakList(out string) []Package {
 		size := parseFlatpakSize(sizeStr)
 
 		pkgs = append(pkgs, Package{
+			ID:       appID,
 			Name:     name,
 			Version:  version,
 			Size:     size,