Installing crowdsec
This was a pretty easy thing to do. Just follow the instructions for debian on their github page
First install their repository into the apt repositories:
$ curl -s https://packagecloud.io/install/repositories/crowdsec/crowdsec/script.deb.sh | sudo bash
[sudo] password for <omitted>:
Detected operating system as Ubuntu/focal.
Checking for curl...
Detected curl...
Checking for gpg...
Detected gpg...
Detected apt version as 2.0.9
Running apt-get update... done.
Installing apt-transport-https... done.
Installing /etc/apt/sources.list.d/crowdsec_crowdsec.list...done.
Importing packagecloud gpg key... Packagecloud gpg key imported to /etc/apt/keyrings/crowdsec_crowdsec-archive-keyring.gpg
done.
Running apt-get update... done.
The repository is setup! You can now install packages.
Then install the package from their apt repository
$ sudo apt-get install crowdsec
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
crowdsec
0 upgraded, 1 newly installed, 0 to remove and 111 not upgraded.
Need to get 71.3 MB of archives.
After this operation, 261 MB of additional disk space will be used.
Get:1 https://packagecloud.io/crowdsec/crowdsec/ubuntu focal/main amd64 crowdsec amd64 1.6.4 [71.3 MB]
Fetched 71.3 MB in 7s (10.2 MB/s)
Preconfiguring packages ...
Selecting previously unselected package crowdsec.
(Reading database ... 214427 files and directories currently installed.)
Preparing to unpack .../crowdsec_1.6.4_amd64.deb ...
You can always run the configuration again interactively by using '/usr/share/crowdsec/wizard.sh -c'
Unpacking crowdsec (1.6.4) ...
Setting up crowdsec (1.6.4) ...
Creating /etc/crowdsec/acquis.yaml
INFO[2025-01-02 16:12:19] crowdsec_wizard: service 'ssh': /var/log/auth.log
INFO[2025-01-02 16:12:19] crowdsec_wizard: using journald for 'smb'
INFO[2025-01-02 16:12:19] crowdsec_wizard: service 'linux': /var/log/syslog /var/log/kern.log
Machine '73c9e33c34494d6ba2f133e0a743443eszG4DHGKBqyr7ppI' successfully added to the local API.
API credentials written to '/etc/crowdsec/local_api_credentials.yaml'.
Updating hub
INFO Wrote index to /etc/crowdsec/hub/.index.json
INFO[2025-01-02 16:12:23] crowdsec_wizard: Installing collection 'crowdsecurity/linux'
updated /var/lib/crowdsec/data/GeoLite2-City.mmdb
updated /var/lib/crowdsec/data/GeoLite2-ASN.mmdb
installed crowdsecurity/linux
INFO[2025-01-02 16:12:31] crowdsec_wizard: Installing collection 'crowdsecurity/smb'
installed crowdsecurity/smb
installed crowdsecurity/whitelists
Created symlink /etc/systemd/system/multi-user.target.wants/crowdsec.service → /lib/systemd/system/crowdsec.service.
Get started with CrowdSec:
* Detailed guides are available in our documentation: https://docs.crowdsec.net
* Configuration items created by the community can be found at the Hub: https://hub.crowdsec.net
* Gain insights into your use of CrowdSec with the help of the console https://app.crowdsec.net
You can always run the configuration again interactively by using '/usr/share/crowdsec/wizard.sh -c'
One of the key lines here is
You can always run the configuration again interactively by using '/usr/share/crowdsec/wizard.sh -c'
Looking around I found how to see what plugins are available for parsing logs
$ sudo cscli parsers list -a
PARSERS
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Name 📦 Status Version Local Path
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
a1ad/meshcentral-logs 🚫 disabled,update-available
a1ad/mikrotik-logs 🚫 disabled,update-available
aderumier/proxmox-iptables-logs 🚫 disabled,update-available
aidalinfo/couchdb-logs 🚫 disabled,update-available
aidalinfo/tcpudp-flood-traefik 🚫 disabled,update-available
andreasbrett/baikal-logs 🚫 disabled,update-available
andreasbrett/paperless-ngx-logs 🚫 disabled,update-available
andreasbrett/webmin-logs 🚫 disabled,update-available
baudneo/gotify-logs 🚫 disabled,update-available
baudneo/zoneminder-logs 🚫 disabled,update-available
corvese/apache-guacamole-logs 🚫 disabled,update-available
crowdsecurity/amavis-logs 🚫 disabled,update-available
crowdsecurity/apache2-logs 🚫 disabled,update-available
crowdsecurity/appsec-logs 🚫 disabled,update-available
crowdsecurity/asterisk-logs 🚫 disabled,update-available
crowdsecurity/auditd-logs 🚫 disabled,update-available
crowdsecurity/aws-cloudfront 🚫 disabled,update-available
crowdsecurity/aws-cloudtrail 🚫 disabled,update-available
crowdsecurity/caddy-logs 🚫 disabled,update-available
crowdsecurity/configserver-lfd-logs 🚫 disabled,update-available
crowdsecurity/cowrie-logs 🚫 disabled,update-available
crowdsecurity/cpanel-logs 🚫 disabled,update-available
crowdsecurity/cri-logs 🚫 disabled,update-available
crowdsecurity/dateparse-enrich ✔️ enabled 0.2 /etc/crowdsec/parsers/s02-enrich/dateparse-enrich.yaml
crowdsecurity/docker-logs 🚫 disabled,update-available
crowdsecurity/dovecot-logs 🚫 disabled,update-available
crowdsecurity/dropbear-logs 🚫 disabled,update-available
crowdsecurity/endlessh-logs 🚫 disabled,update-available
crowdsecurity/exchange-imap-logs 🚫 disabled,update-available
crowdsecurity/exchange-pop-logs 🚫 disabled,update-available
crowdsecurity/exchange-smtp-logs 🚫 disabled,update-available
crowdsecurity/exim-logs 🚫 disabled,update-available
crowdsecurity/fastly-logs 🚫 disabled,update-available
crowdsecurity/fortinet-logs 🚫 disabled,update-available
crowdsecurity/freeswitch 🚫 disabled,update-available
crowdsecurity/geoip-enrich ✔️ enabled 0.5 /etc/crowdsec/parsers/s02-enrich/geoip-enrich.yaml
crowdsecurity/haproxy-logs 🚫 disabled,update-available
crowdsecurity/home-assistant-logs 🚫 disabled,update-available
crowdsecurity/http-logs 🚫 disabled,update-available
crowdsecurity/iis-logs 🚫 disabled,update-available
crowdsecurity/iptables-logs 🚫 disabled,update-available
crowdsecurity/jellyfin-whitelist 🚫 disabled,update-available
crowdsecurity/k8s-audit 🚫 disabled,update-available
crowdsecurity/kasm-logs 🚫 disabled,update-available
crowdsecurity/laurel-logs 🚫 disabled,update-available
crowdsecurity/litespeed-logs 🚫 disabled,update-available
crowdsecurity/magento-extension-logs 🚫 disabled,update-available
crowdsecurity/mariadb-logs 🚫 disabled,update-available
crowdsecurity/modsecurity 🚫 disabled,update-available
crowdsecurity/mssql-logs 🚫 disabled,update-available
crowdsecurity/mysql-logs 🚫 disabled,update-available
crowdsecurity/naxsi-logs 🚫 disabled,update-available
crowdsecurity/nextcloud-logs 🚫 disabled,update-available
crowdsecurity/nextcloud-whitelist 🚫 disabled,update-available
crowdsecurity/nginx-logs 🚫 disabled,update-available
crowdsecurity/nginx-proxy-manager-logs 🚫 disabled,update-available
crowdsecurity/odoo-logs 🚫 disabled,update-available
crowdsecurity/opnsense-gui-logs 🚫 disabled,update-available
crowdsecurity/palo-alto-threat-log 🚫 disabled,update-available
crowdsecurity/pam-logs 🚫 disabled,update-available
crowdsecurity/pfsense-gui-logs 🚫 disabled,update-available
crowdsecurity/pgsql-logs 🚫 disabled,update-available
crowdsecurity/pkexec-logs 🚫 disabled,update-available
crowdsecurity/plex-allowlist 🚫 disabled,update-available
crowdsecurity/postfix-logs 🚫 disabled,update-available
crowdsecurity/postscreen-logs 🚫 disabled,update-available
crowdsecurity/proftpd-logs 🚫 disabled,update-available
crowdsecurity/sabnzbd-logs 🚫 disabled,update-available
crowdsecurity/segfault-logs 🚫 disabled,update-available
crowdsecurity/smb-logs ✔️ enabled 0.2 /etc/crowdsec/parsers/s01-parse/smb-logs.yaml
crowdsecurity/sshd-logs ✔️ enabled 2.8 /etc/crowdsec/parsers/s01-parse/sshd-logs.yaml
crowdsecurity/sshd-success-logs 🚫 disabled,update-available
crowdsecurity/stirling-pdf-logs 🚫 disabled,update-available
crowdsecurity/supabase-docker-pgsql 🚫 disabled,update-available
crowdsecurity/suricata-logs 🚫 disabled,update-available
crowdsecurity/synology-dsm-logs 🚫 disabled,update-available
crowdsecurity/syslog-logs ✔️ enabled 0.8 /etc/crowdsec/parsers/s00-raw/syslog-logs.yaml
crowdsecurity/sysmon-logs 🚫 disabled,update-available
crowdsecurity/tcpdump-logs 🚫 disabled,update-available
crowdsecurity/teamspeak3-logs 🚫 disabled,update-available
crowdsecurity/teleport-logs 🚫 disabled,update-available
crowdsecurity/thehive-logs 🚫 disabled,update-available
crowdsecurity/traefik-logs 🚫 disabled,update-available
crowdsecurity/unifi-logs 🚫 disabled,update-available
crowdsecurity/vsftpd-logs 🚫 disabled,update-available
crowdsecurity/whitelists ✔️ enabled 0.2 /etc/crowdsec/parsers/s02-enrich/whitelists.yaml
crowdsecurity/windows-auth 🚫 disabled,update-available
crowdsecurity/windows-firewall-logs 🚫 disabled,update-available
crowdsecurity/windows-logs 🚫 disabled,update-available
crowdsecurity/wireguard-logs 🚫 disabled,update-available
darkclip/charon-ipsec-logs 🚫 disabled,update-available
Dominic-Wagner/vaultwarden-logs 🚫 disabled,update-available
firewallservices/lemonldap-ng 🚫 disabled,update-available
firewallservices/pf-logs 🚫 disabled,update-available
firewallservices/zimbra-logs 🚫 disabled,update-available
firix/authentik-logs 🚫 disabled,update-available
fulljackz/proxmox-logs 🚫 disabled,update-available
fulljackz/pureftpd-logs 🚫 disabled,update-available
gauth-fr/immich-logs 🚫 disabled,update-available
hitech95/nginx-mail-logs 🚫 disabled,update-available
inherent-io/keycloak-logs 🚫 disabled,update-available
jbowdre/miniflux-logs 🚫 disabled,update-available
jusabatier/apereo-cas-audit-logs 🚫 disabled,update-available
LePresidente/adguardhome-logs 🚫 disabled,update-available
LePresidente/authelia-logs 🚫 disabled,update-available
LePresidente/emby-logs 🚫 disabled,update-available
LePresidente/gitea-logs 🚫 disabled,update-available
LePresidente/grafana-logs 🚫 disabled,update-available
LePresidente/harbor-logs 🚫 disabled,update-available
LePresidente/jellyfin-logs 🚫 disabled,update-available
LePresidente/jellyseerr-logs 🚫 disabled,update-available
LePresidente/ombi-logs 🚫 disabled,update-available
LePresidente/overseerr-logs 🚫 disabled,update-available
LePresidente/redmine-logs 🚫 disabled,update-available
lourys/pterodactyl-wings-logs 🚫 disabled,update-available
MariuszKociubinski/bitwarden-logs 🚫 disabled,update-available
mstilkerich/bind9-logs 🚫 disabled,update-available
mwinters-stuff/mailu-admin-logs 🚫 disabled,update-available
openappsec/openappsec-logs 🚫 disabled,update-available
plague-doctor/audiobookshelf-logs 🚫 disabled,update-available
pserranoa/openvpn 🚫 disabled,update-available
schiz0phr3ne/prowlarr-logs 🚫 disabled,update-available
schiz0phr3ne/radarr-logs 🚫 disabled,update-available
schiz0phr3ne/sonarr-logs 🚫 disabled,update-available
thespad/sshesame-logs 🚫 disabled,update-available
timokoessler/gitlab-logs 🚫 disabled,update-available
timokoessler/mongodb-logs 🚫 disabled,update-available
timokoessler/uptime-kuma-logs 🚫 disabled,update-available
xs539/bookstack-logs 🚫 disabled,update-available
xs539/joplin-server-logs 🚫 disabled,update-available
Zaulao/aws-alb 🚫 disabled,update-available
ZoeyVid/npmplus-logs 🚫 disabled,update-available
Then checking the metrics I saw that nothing is getting parsed:
$ sudo cscli metrics show acquisition
Acquisition Metrics:
╭─────────────────────────────────────────────────┬────────────┬──────────────┬────────────────┬────────────────────────┬───────────────────╮
│ Source │ Lines read │ Lines parsed │ Lines unparsed │ Lines poured to bucket │ Lines whitelisted │
├─────────────────────────────────────────────────┼────────────┼──────────────┼────────────────┼────────────────────────┼───────────────────┤
│ file:/var/log/auth.log │ 58 │ - │ 58 │ - │ - │
│ file:/var/log/syslog │ 9 │ - │ 9 │ - │ - │
│ journalctl:journalctl-_SYSTEMD_UNIT=smb.service │ 1 │ - │ 1 │ - │ - │
╰─────────────────────────────────────────────────┴────────────┴──────────────┴────────────────┴────────────────────────┴───────────────────╯
The answer to this was in this help article
Basically, crowdsec only records parsed lines that it recognizes. Other lines are not parsed. So I used another terminal session to login and purposely failed the login.
$ ssh plex-server
Enter passphrase for key '/mnt/c/Users/<omitted>/.ssh/id_rsa':
<omitted>@<omitted>'s password:
Permission denied, please try again.
<omitted>@<omitted>'s password:
Permission denied, please try again.
<omitted>@<omitted>'s password:
<omitted>@<omitted>: Permission denied (publickey,password).
Now, upon checking the metrics, I now see that lines in the log are parsed:
$ sudo cscli metrics show acquisition
Acquisition Metrics:
╭─────────────────────────────────────────────────┬────────────┬──────────────┬────────────────┬────────────────────────┬───────────────────╮
│ Source │ Lines read │ Lines parsed │ Lines unparsed │ Lines poured to bucket │ Lines whitelisted │
├─────────────────────────────────────────────────┼────────────┼──────────────┼────────────────┼────────────────────────┼───────────────────┤
│ file:/var/log/auth.log │ 70 │ 3 │ 67 │ - │ 3 │
│ file:/var/log/syslog │ 9 │ - │ 9 │ - │ - │
│ journalctl:journalctl-_SYSTEMD_UNIT=smb.service │ 1 │ - │ 1 │ - │ - │
╰─────────────────────────────────────────────────┴────────────┴──────────────┴────────────────┴────────────────────────┴───────────────────╯
So it is all setup. It found my smb service, my sshd service, and that's all I have running. If I install nginx or something in the future I can enable those plugins as well by running the wizard or doing it manually. Overall this was pretty painless. Next thing is to check out their web console and install that, but I'll leave this running for a while before I investigate that service.