Como outros já apontaram, você pode usar o apt-get com as opções --purge remove e autoremove / autoclean.
Você também pode usar o comando deborphan (um pacote separado) para identificar pacotes órfãos e excluir os pacotes identificados para:
sudo apt-get --purge remove `deborphan`
Um programa mais abrangente é o debfoster (que remove pacotes Debian desnecessários) que eu corro frequentemente para manter meu sistema arrumado. Um HOWTO bastante antigo no debfoster mostra como se livrar de todos os pacotes do KDE com instruções que ainda são relevantes.
Um pequeno trecho de debfoster em operação (recentemente, usei-o para limpar o eclipse e pacotes relacionados do meu sistema). Observe que você pode passar um pacote para focar como parâmetro.
Eu regularmente tento descartar software, e o debfoster é incrivelmente útil para garantir o mínimo de inchaço dos pacotes indesejados.
$ sudo debfoster
ubuntu-desktop is keeping the following 194 packages installed:
acpi-support acpid apmd app-install-data-partner apport apport-gtk at-spi avahi-autoipd bluez-utils brltty brltty-x11
checkbox checkbox-gtk compiz compiz-fusion-plugins-extra compiz-fusion-plugins-main contact-lookup-applet cupsddk
cupsddk-drivers dc dcraw dmz-cursor-theme dnsmasq-base ekiga eog ethtool evolution-exchange evolution-indicator
example-content fast-user-switch-applet fglrx-modaliases file-roller finger firefox-3.0-gnome-support
firefox-gnome-support foo2zjs foomatic-db-hpijs gcalctool gdm gdm-guest-session gedit gedit-common
gnome-accessibility-themes gnome-cards-data gnome-codec-install gnome-games gnome-games-data gnome-mag gnome-nettool
gnome-orca gnome-power-manager gnome-screensaver gnome-session-canberra gnome-system-tools gnome-themes-selected
gnome-themes-ubuntu gstreamer0.10-plugins-base-apps gstreamer0.10-schroedinger gstreamer0.10-tools gtk2-engines
gtk2-engines-murrine gucharmap gvfs-fuse hal-cups-utils hotkey-setup hpijs hplip hplip-data human-icon-theme human-theme
im-switch inputattach jockey-common jockey-gtk language-selector language-selector-common laptop-mode-tools lftp
libavahi-ui0 libbrlapi0.5 libcolamd-3.2.0 libcryptui0 libdmx1 libggz2 libggzcore9 libggzmod4 libgmime2.2a-cil
libgnome-mag2 libgnomepanel2.24-cil libgnomevfs2-bin libgtk-vnc-1.0-0 libieee1284-3 libmbca0 libnm-glib0 libnm-util1
libopal3.6.1 libotr2 libpt2.6.1 libpt2.6.1-plugins-alsa libpt2.6.1-plugins-v4l2 libsane libscim8c2a libsnmp-base libsnmp15
libusplash0 libwmf0.2-7-gtk libwpg-0.1-1 linux-headers-2.6.28-15 linux-headers-2.6.28-15-generic linux-headers-generic
lp-solve min12xxw mobile-broadband-provider-info mscompress nautilus-sendto nautilus-share network-manager
network-manager-gnome notify-osd nvidia-173-modaliases nvidia-180-modaliases nvidia-71-modaliases nvidia-96-modaliases
nvidia-common onboard openoffice.org-calc openoffice.org-draw openoffice.org-gnome openoffice.org-gtk
openoffice.org-impress openprinting-ppds pcmciautils pidgin pidgin-libnotify pidgin-otr pnm2ppa pxljr python-brlapi
python-cups python-cupshelpers python-gtksourceview2 python-pyatspi python-smbc python-usb python-virtkey readahead
rss-glx sane-utils scim scim-bridge-agent scim-bridge-client-gtk scim-gtk2-immodule scim-modules-socket
screensaver-default-images seahorse seahorse-plugins splix ssh-askpass-gnome syslinux system-config-printer-common
system-config-printer-gnome tangerine-icon-theme tomboy toshset tsclient ttf-arabeyes ttf-arphic-uming ttf-lao
ttf-sazanami-gothic ttf-thai-tlwg ubuntu-artwork ubuntu-docs ubuntu-gdm-themes ubuntu-sounds ubuntu-wallpapers
update-manager update-notifier usb-creator usplash usplash-theme-ubuntu vinagre vino whois wireless-tools wpasupplicant
xcursor-themes xdg-user-dirs-gtk xfonts-100dpi xfonts-75dpi xfonts-scalable xinput xorg xscreensaver-data xscreensaver-gl
Keep ubuntu-desktop? [Ynpsiuqx?], [H]elp: Y
eclipse-cdt is keeping the following 77 packages installed:
ant ant-gcj ant-optional ant-optional-gcj antlr default-jdk default-jre ecj ecj-gcj eclipse eclipse-gcj eclipse-jdt
eclipse-jdt-gcj eclipse-pde eclipse-pde-gcj eclipse-platform eclipse-platform-gcj eclipse-rcp eclipse-rcp-gcj
eclipse-source fastjar gappletviewer-4.3 gcj-4.2-base gcj-4.3 gcj-4.3-base gij-4.2 gij-4.3 gjdoc java-gcj-compat
java-gcj-compat-dev java-gcj-compat-headless junit junit4 libantlr-java libantlr-java-gcj libbcel-java
libcommons-beanutils-java libcommons-collections-java libcommons-collections3-java libcommons-dbcp-java
libcommons-digester-java libcommons-el-java libcommons-launcher-java libcommons-logging-java libcommons-modeler-java
libcommons-pool-java libecj-java libecj-java-gcj libgcj-bc libgcj-common libgcj8-1 libgcj8-1-awt libgcj8-jar libgcj9-0
libgcj9-0-awt libgcj9-dev libgcj9-jar libgcj9-src libjaxp1.3-java libjaxp1.3-java-gcj libjsch-java liblog4j1.2-java
liblog4j1.2-java-gcj liblucene-java liblucene-java-doc libmx4j-java libregexp-java libservlet2.3-java libservlet2.4-java
libswt3.2-gtk-gcj libswt3.2-gtk-java libswt3.2-gtk-jni libtomcat5.5-java libxerces2-java libxerces2-java-gcj libxt-dev
openjdk-6-jdk
Keep eclipse-cdt? [Ynpsiuqx?], [H]elp: H
Yes Keep eclipse-cdt. [default]
No Delete eclipse-cdt.
Prune Delete eclipse-cdt and the packages it is keeping installed.
Skip Skip this question.
Help Print this message.
Info or ? Show information about eclipse-cdt.
Undo Undo last response.
Quit Exit without removing packages.
Exit Remove unwanted packages and exit.
Keep eclipse-cdt? [Ynpsiuqx?], [H]elp: P
libpanelappletmm-2.6-dev is keeping the following 9 packages installed:
libcairomm-1.0-dev libgconfmm-2.6-dev libglibmm-2.4-dev libgnomemm-2.6-dev libgtkmm-2.4-dev libpanel-applet2-dev
libpanelappletmm-2.6-1c2 libpangomm-1.4-dev libsigc++-2.0-dev
Keep libpanelappletmm-2.6-dev? [Ynpsiuqx?], [H]elp: S
Edit: Um bom link para aprender sobre o uso de deborphan. Uma explicação concisa do debfoster.