Falco

Falco is a runtime threat detection engine

Falco consumes events from different sources: system calls, Kubernetes Audit Events, and Cloud Activity Logs (via Falco Plugins, to ingest AWS events from Cloudtrail)

Installation

Drivers needed

Falco needs a driver (the kernel module or the eBPF probe) to work; if a prebuilt driver is not available for your distribution/kernel, Falco needs kernel headers installed on the host to build the driver on the fly.

falco-driver-check is a script for checking if Falco drivers are available for your system.

Driver	Description
Kernel Module	Standard module to interact with the kernel
eBPF probe	Use the eBPF probe when loading a kernel module is not a viable option If you are running Falco on GKE or ARM/Raspberry Pis, it is recommended to use the eBPF probe. But, as a rule of thumb, always use the Kernel Module
Userspace instrumentation (pdig)	Operates entirely in the userspace using ptrace and pdig Use pdig in environments where kernel modules and eBPF are not an option, such as artifacts with no access to the host (e.g., AWS ECS/Fargate and other containers as a service)

Docker

• There are 2 container images:
  • The standard Falco container image, which will try to install the driver on the host
  • The no-driver image, which requires the driver already on the host
• Falco requires its kernel module installed directly on the host system
  • You can use the standard installation method directly on the host
  • Alternatively, you can temporarily use a privileged container to install the driver

# Temporarily use a privileged container to install the driver on the host
$ docker pull falcosecurity/falco-driver-loader:latest
$ docker run --rm -i -t \
    --privileged \
    -v /root/.falco:/root/.falco \
    -v /proc:/host/proc:ro \
    -v /boot:/host/boot:ro \
    -v /lib/modules:/host/lib/modules:ro \
    -v /usr:/host/usr:ro \
    -v /etc:/host/etc:ro \
    falcosecurity/falco-driver-loader:latest && echo "Falco drivers installed!"

# Run Falco in a container with the principle of least privilege:
$ docker pull falcosecurity/falco-no-driver:latest
$ docker run --rm -d \
    --name "falco_daemon" \
    --security-opt apparmor:unconfined \
    -p 8765:8765 \
    -e HOST_ROOT=/ \
    --cap-add SYS_PTRACE \
    --pid=host $(ls /dev/falco* | xargs -I {} echo --device {}) \
    -v /var/run/docker.sock:/var/run/docker.sock \
    falcosecurity/falco-no-driver:latest

Kubernetes

1. Deployment
   • The most secure way to run Falco is to install Falco directly on the host system, so that Falco is isolated from Kubernetes in case of compromise
   • Falco can also be run directly in Kubernetes, if isolation is not a concern

     $ helm repo add falcosecurity https://falcosecurity.github.io/charts
     $ helm repo update
     $ helm install falco                         \
           --create-namespace                     \
           --namespace falco                      \
           --set ebpf.enabled=true                \
           --set auditLog.enabled=true            \
           --set falcosidekick.enabled=true       \
           --set falcosidekick.webui.enabled=true \
           falcosecurity/falco
     

2. Verification
   1. Falco has a webserver that captures K8S events:

      # Review the status of the Falco web server
      # The desired output {"status": "ok"} confirms that the Falco web server is running as expected
      $ kubectl exec "$container" -- curl -s localhost:8765/healthz; echo
      

   2. Check with source of events is configured

      $ kubectl logs -n falco -l "app.kubernetes.io/name=falco" -c falco-driver-loader --tail=-1 | grep "* Running falco-driver-loader with"
      
      # Output for kernel module
      * Running falco-driver-loader with: driver=module, compile=yes, download=yes
      * Running falco-driver-loader with: driver=module, compile=yes, download=yes
      
      # Output for eBPF
      * Running falco-driver-loader with: driver=bpf, compile=yes, download=yes
      * Running falco-driver-loader with: driver=bpf, compile=yes, download=yes
      

   3. Confirm the driver is properly installed:

      $ kubectl logs -n falco -l "app.kubernetes.io/name=falco" -c falco-driver-loader --tail=-1 | grep -A 5 "* Success"
      

   4. Trigger one of the Falco rules:

      $ export POD_NAME=$(kubectl get pods --namespace falco -l "app.kubernetes.io/name=falco" -o jsonpath="{.items[0].metadata.name}")
      $ kubectl -n falco exec ${POD_NAME} -- find /root -name "id_rsa"
      
      # Check that Falco correctly intercepted the potentially dangerous command:
      $ kubectl logs -n falco -l "app.kubernetes.io/name=falco" | grep Warning
      

Additional setup for self-hosted Kubernetes

Configure the K8s audit policy:

1. Download the policy

   $ wget https://raw.githubusercontent.com/falcosecurity/evolution/master/examples/k8s_audit_config/audit-policy.yaml
   

2. Define and deploy the webhook audit backend

   # This manifest defines how to send audit events to a remote web API
   # (in this case, the Falco service ingesting the K8s audit logs)
   $ export IP=$(kubectl get service --namespace falco falco -o=jsonpath={.spec.clusterIP})
   $ cat << EOF | sudo tee webhook-config.yaml
   apiVersion: v1
   kind: Config
   clusters:
   - name: falco
     cluster:
       server: http://$IP:8765/k8s-audit
   contexts:
   - context:
       cluster: falco
       user: ""
     name: default-context
   current-context: default-context
   preferences: {}
   users: []
   EOF
   

3. Patch the kube-apiserver using the flags audit-log-path, audit-policy-file, and audit-webhook-config-file

Falco Rules

Rule files

• Default rules file: /etc/falco/falco_rules.yaml
  • Not to be modified
  • Contains a predefined set of rules designed to provide good coverage in a variety of situations
  • Show the name and description of all rules: kubectl exec ${FALCO_POD} -n falco -- falco -L
• Local rules file: /etc/falco/falco_rules.local.yaml
  • For additions/overrides/modifications

Main Constructs

A Falco rules file is a YAML file containing three types of elements:

Element	Description	Fields
rule	A condition under which an alert should be generated, and the output string to send with the alert	rule: A short, unique name for the rule desc: A longer description of what the rule detects condition: A filtering expression that is applied against events to check whether they match the rule output: Message to output if a matching event occurs priority: The severity of the event (EMERGENCY, ALERT, CRITICAL, ERROR, WARNING, NOTICE, INFORMATIONAL, DEBUG) exceptions: A set of exceptions that will prevent a rule from triggering an alert enabled: If set to false, a rule is neither loaded nor matched against any events tags: A list of tags applied to the rule
macro	A rule condition snippet that can be re-used inside rules and other macros	macro: Name of the macro condition: Filtering expression, same as used in rules
list	Named collections of items that can be included in rules, macros, or other lists	list: Name of the List items: List of values (inside square brackets and separated by comma)

Rule exampleMacro exampleLists example

- rule: shell_in_container
  desc: notice shell activity within a container
  condition: container.id != host and proc.name = bash
  output: shell in a container (user=%user.name container_name=%container.name)
  priority: warning
  tags: [shell, container]

- macro: in_container
  condition: container.id != host

- macro: spawned_shell
  condition: proc.name = bash

- rule: shell_in_container
  desc: notice shell activity within a container
  condition: spawned_shell and in_container
  output: shell in a container (user=%user.name ...)
  priority: WARNING

- list: shell_binaries
  items: [bash, csh, ksh, sh, tcsh, zsh, dash]

- list: userexec_binaries
  items: [sudo, su]

- list: known_binaries
  items: [shell_binaries, userexec_binaries]

- macro: safe_procs
  condition: proc.name in (known_binaries)

Field Classes

Description

• Falco events expose many different fields, organized in field classes
• Falco enriches events with metadata from the available contexts. Not all field classes or fields are available for all the events

Field Class	Description
evt	Generic event fields For syscall events, the evt.type field is the name of the syscall The evt.dir field (known as the "direction") can have two values: > which indicates entry (the call is invoked) → e.g., > setuid(UID uid) < which indicates exit (the call has returned) → e.g., < setuid(ERRNO res)
process	Additional information about the process and thread executing the syscall event
user	Information about the user executing the specific event
fd	Every syscall that has a file descriptor in its arguments has these fields set with information related to the file
container	Container information If the event is not happening inside a container, both id and name will be set to host
k8s	Kubernetes related context

Manipulation

Action	Description
Appending	Rules/Macros/Lists can be extended with append Write the element using the same name Add the append key set to true Add the appended items (list) or conditions (macro or rule) Main use cases: Tune noisy rules that generate false positives Add additional items to lists or macros that are custom to your environment Where appended: When appending lists, items are added to the end of the list When appending rules or macros, the additional text is appended to the condition field
Override macros	Some of the default rules are a placeholder which exist to be customized To customize these rules, write to your local rules file a macro with the same name
Enable/Disable default rules	By using Falco parameters: -D <substring>: Disable any rules with names having the specified substring -T <tag>: Disable any rules that contains the specified tag -t <tag>: Only run those rules that contain the specified tag To specify in Helm charts: --set "extraArgs={-D <substring>}" By using existing macros: Most of the default rules offer some kind of consider_* macros, which are already part of the rule conditions These macros are usually set to (never_true) or (always_true) By using custom rules definitions: Sometimes consider_* is not available An alternative is to use the append rule properties

Append exampleOverride macros exampleEnable rules example

# There is a rule in falco_rules.yaml
- list: sensitive_directory_names
  items: [/, /etc, /etc/, /root, /root/]

# Add the following list definition to the my_rules.yaml file
- list: sensitive_directory_names
  append: true
  items: [/mnt]

# Original macro in falco_rules.yaml:
- macro: user_trusted_containers
  condition: (never_true)

# Overriding macro in falco_rules.local.yaml:
- macro: user_trusted_containers
  condition: (container.image startswith sysdig/agent)

• Enable via existing macro:

  # Disable the default rule "Packet socket created in container"
  - macro: consider_packet_socket_communication
  condition: (always_true)
  

• Enable via custom rule definitions:

  - rule: User mgmt binaries
  append: true
  condition: and (never_true)
  

Exceptions

There are multiple ways to deal with noisy alerts:

Method	Description
Disabling one or more default rules (not recommended)	Disabling the entire rule reduces the detection scope of Falco
Modifying a default rule condition	Concatenate one or more conditions to the default rule to exclude a particular process, file descriptor or any other pair of field-value that identifies the exception
Defining rule exceptions	If a particular rule requires tuning, an exception can be defined

Generic ExampleRun shell untrusted

- rule: <the_name_of_the_rule>
  desc: (...)
  condition: (...)
  output: (...)
  tags: (...)
  exceptions:
  - name: <name_of_the_exception>
    fields: [proc.name, fd.name]
    comps: [=, in]
    values:
      - [my_bin_A, [my_file_A]]
      - [custom_bin_B, [allowed_file_for_B, other_B_file]]

Exception that removes alerts for the Run shell untrusted rule when:

• proc.pname = apache2
• proc.cmdline = bash -c ls /root > /tmp/pmt

customRules:
  custom_rules_from_default: |-
    (...)
    - rule: Run shell untrusted
    (...)
      exceptions:
      - name: app1_shell_runner
        fields: [proc.pname, proc.cmdline]
        comps: [=, =]
        values:
        - [apache2, bash -c ls /root > /tmp/pmt]
    (...)

Tip

• Fields, comps and values are grouped in tuples
• They are joined/matched following the order in which they are defined.
• Example:
  • The exception below considers proc.name = process_A and fd.name in file_A
  • The first field and the first value are compared using the first operator
  • The second field, with the second operator and value/values

- rule:
  (...)
  exceptions:
  - name: example_exception
    fields: [proc.name, fd.name]
    comps: [=, in]
    values:
      - [process_A, file_A]
      - [process_B, [file_B, file_C]]

Falco Alerts

• When a Falco Rule is violated, an alert is triggered
• Alerts can be sent to multiple supported channels

Channel	Description
Standard Output	Default Errors are printed to stdout/syslog
File Output	Alerts are written to file Example: file_output:     enabled: true     keep_alive: true     filename: ./events.txt
Program output	Redirects Falco alerts to the standard input of a desired command Example (mail): program_output:     enabled: true     keep_alive: true     program: mail -s "Falco Notification" [email protected] Example (Elasticsearch): program_output:     enabled: true     keep_alive: true     program: "curl -s -H 'Content-Type: application/json' -d @- -X POST http://localhost:9200/falco/_doc"
HTTP endpoint	Send Falco alerts to an HTTP[s] URL Example: http_output:     enabled: true     url: http://some.url/some/path/
gRPC client	Not enabled by default The only officially supported service using this output alternative is falco-exporter (for Prometheus)

Falco Response Engine

Falco is a Detection Engine, but it can be upgraded to a Response Engine with the help of Falco Sidekick.

Component	Description
falcosidekick	A aemon for connecting Falco with multiple tools and protocols It takes a stream of Falco events and forward them to different outputs (e.g., chats, log systems, SIEM, etc.) It can also be used as a Response Engine to define mitigation actions to threats detected by Falco Falco Sidekick needs to be connected to another tool that leads to the desired action. Some approaches: Kubeless OpenFaas Knative Tekton Argo Cloud Run Cloud Functions
sample-event-generator	Can generate sample activities for both syscall and k8s audit related rules Installation: helm upgrade --namespace falco \     --set fakeEventGenerator.enabled=true \     --reuse-values falco falcosecurity/falco The tool provides a command to run either some or all sample events: event-generator run [regexp] Activity Description Call System Call Performs a variety of suspect actions that are detected by the default Falco ruleset event-generator run syscall --loop Kubernetes Auditing Generates activity that matches the k8s audit event ruleset event-generator run k8saudit --loop --namespace <namespace>
falco-exporter	Prometheus Metrics Exporter for Falco output events

---

Resources

General

Link	Description
Sysdig Official Docs
Sysdig Falco 101	All you need to learn to get started with Falco
Sysdig Falco Hands-on Labs	Learn and practice with Falco in your Browser
Sysdig Getting started with runtime security and Falco	How to get started with Falco
Analyze Kubernetes Audit logs using Falco	Detect intrusions that happened in your Kubernetes cluster through audit logs using Falco

Rulesets

Link	Description
Using Falco to monitor outbound traffic for Pods in Kubernetes	Walkthrough on how to use Falco to monitor the network traffic of your Pods