Skip to content

Falco

Falco

Falco is a runtime threat detection engine

Falco consumes events from different sources: system calls, Kubernetes Audit Events, and Cloud Activity Logs (via Falco Plugins, to ingest AWS events from Cloudtrail)

Installation

Drivers needed

Falco needs a driver (the kernel module or the eBPF probe) to work; if a prebuilt driver is not available for your distribution/kernel, Falco needs kernel headers installed on the host to build the driver on the fly.

falco-driver-check is a script for checking if Falco drivers are available for your system.

Driver Description
Kernel Module
  • Standard module to interact with the kernel
eBPF probe
  • Use the eBPF probe when loading a kernel module is not a viable option
  • If you are running Falco on GKE or ARM/Raspberry Pis, it is recommended to use the eBPF probe. But, as a rule of thumb, always use the Kernel Module
Userspace instrumentation (pdig)
  • Operates entirely in the userspace using ptrace and pdig
  • Use pdig in environments where kernel modules and eBPF are not an option, such as artifacts with no access to the host (e.g., AWS ECS/Fargate and other containers as a service)
Docker
  • There are 2 container images:
    • The standard Falco container image, which will try to install the driver on the host
    • The no-driver image, which requires the driver already on the host
  • Falco requires its kernel module installed directly on the host system
    • You can use the standard installation method directly on the host
    • Alternatively, you can temporarily use a privileged container to install the driver
# Temporarily use a privileged container to install the driver on the host
$ docker pull falcosecurity/falco-driver-loader:latest
$ docker run --rm -i -t \
    --privileged \
    -v /root/.falco:/root/.falco \
    -v /proc:/host/proc:ro \
    -v /boot:/host/boot:ro \
    -v /lib/modules:/host/lib/modules:ro \
    -v /usr:/host/usr:ro \
    -v /etc:/host/etc:ro \
    falcosecurity/falco-driver-loader:latest && echo "Falco drivers installed!"

# Run Falco in a container with the principle of least privilege:
$ docker pull falcosecurity/falco-no-driver:latest
$ docker run --rm -d \
    --name "falco_daemon" \
    --security-opt apparmor:unconfined \
    -p 8765:8765 \
    -e HOST_ROOT=/ \
    --cap-add SYS_PTRACE \
    --pid=host $(ls /dev/falco* | xargs -I {} echo --device {}) \
    -v /var/run/docker.sock:/var/run/docker.sock \
    falcosecurity/falco-no-driver:latest
Kubernetes
  1. Deployment
    • The most secure way to run Falco is to install Falco directly on the host system, so that Falco is isolated from Kubernetes in case of compromise
    • Falco can also be run directly in Kubernetes, if isolation is not a concern
      $ helm repo add falcosecurity https://falcosecurity.github.io/charts
      $ helm repo update
      $ helm install falco                         \
            --create-namespace                     \
            --namespace falco                      \
            --set ebpf.enabled=true                \
            --set auditLog.enabled=true            \
            --set falcosidekick.enabled=true       \
            --set falcosidekick.webui.enabled=true \
            falcosecurity/falco
      
  2. Verification
    1. Falco has a webserver that captures K8S events:
      # Review the status of the Falco web server
      # The desired output {"status": "ok"} confirms that the Falco web server is running as expected
      $ kubectl exec "$container" -- curl -s localhost:8765/healthz; echo
      
    2. Check with source of events is configured
      $ kubectl logs -n falco -l "app.kubernetes.io/name=falco" -c falco-driver-loader --tail=-1 | grep "* Running falco-driver-loader with"
      
      # Output for kernel module
      * Running falco-driver-loader with: driver=module, compile=yes, download=yes
      * Running falco-driver-loader with: driver=module, compile=yes, download=yes
      
      # Output for eBPF
      * Running falco-driver-loader with: driver=bpf, compile=yes, download=yes
      * Running falco-driver-loader with: driver=bpf, compile=yes, download=yes
      
    3. Confirm the driver is properly installed:
      $ kubectl logs -n falco -l "app.kubernetes.io/name=falco" -c falco-driver-loader --tail=-1 | grep -A 5 "* Success"
      
    4. Trigger one of the Falco rules:
      $ export POD_NAME=$(kubectl get pods --namespace falco -l "app.kubernetes.io/name=falco" -o jsonpath="{.items[0].metadata.name}")
      $ kubectl -n falco exec ${POD_NAME} -- find /root -name "id_rsa"
      
      # Check that Falco correctly intercepted the potentially dangerous command:
      $ kubectl logs -n falco -l "app.kubernetes.io/name=falco" | grep Warning
      
Additional setup for self-hosted Kubernetes

Configure the K8s audit policy:

  1. Download the policy
    $ wget https://raw.githubusercontent.com/falcosecurity/evolution/master/examples/k8s_audit_config/audit-policy.yaml
    
  2. Define and deploy the webhook audit backend
    # This manifest defines how to send audit events to a remote web API
    # (in this case, the Falco service ingesting the K8s audit logs)
    $ export IP=$(kubectl get service --namespace falco falco -o=jsonpath={.spec.clusterIP})
    $ cat << EOF | sudo tee webhook-config.yaml
    apiVersion: v1
    kind: Config
    clusters:
    - name: falco
      cluster:
        server: http://$IP:8765/k8s-audit
    contexts:
    - context:
        cluster: falco
        user: ""
      name: default-context
    current-context: default-context
    preferences: {}
    users: []
    EOF
    
  3. Patch the kube-apiserver using the flags audit-log-path, audit-policy-file, and audit-webhook-config-file

Falco Rules

Rule files

  • Default rules file: /etc/falco/falco_rules.yaml
    • Not to be modified
    • Contains a predefined set of rules designed to provide good coverage in a variety of situations
    • Show the name and description of all rules: kubectl exec ${FALCO_POD} -n falco -- falco -L
  • Local rules file: /etc/falco/falco_rules.local.yaml
    • For additions/overrides/modifications

Main Constructs

A Falco rules file is a YAML file containing three types of elements:

Element Description Fields
rule A condition under which an alert should be generated, and the output string to send with the alert
  • rule: A short, unique name for the rule
  • desc: A longer description of what the rule detects
  • condition: A filtering expression that is applied against events to check whether they match the rule
  • output: Message to output if a matching event occurs
  • priority: The severity of the event (EMERGENCY, ALERT, CRITICAL, ERROR, WARNING, NOTICE, INFORMATIONAL, DEBUG)
  • exceptions: A set of exceptions that will prevent a rule from triggering an alert
  • enabled: If set to false, a rule is neither loaded nor matched against any events
  • tags: A list of tags applied to the rule
macro A rule condition snippet that can be re-used inside rules and other macros
  • macro: Name of the macro
  • condition: Filtering expression, same as used in rules
list Named collections of items that can be included in rules, macros, or other lists
  • list: Name of the List
  • items: List of values (inside square brackets and separated by comma)
- rule: shell_in_container
  desc: notice shell activity within a container
  condition: container.id != host and proc.name = bash
  output: shell in a container (user=%user.name container_name=%container.name)
  priority: warning
  tags: [shell, container]
- macro: in_container
  condition: container.id != host

- macro: spawned_shell
  condition: proc.name = bash

- rule: shell_in_container
  desc: notice shell activity within a container
  condition: spawned_shell and in_container
  output: shell in a container (user=%user.name ...)
  priority: WARNING
- list: shell_binaries
  items: [bash, csh, ksh, sh, tcsh, zsh, dash]

- list: userexec_binaries
  items: [sudo, su]

- list: known_binaries
  items: [shell_binaries, userexec_binaries]

- macro: safe_procs
  condition: proc.name in (known_binaries)

Field Classes

Description
  • Falco events expose many different fields, organized in field classes
  • Falco enriches events with metadata from the available contexts. Not all field classes or fields are available for all the events
Field Class Description
evt
  • Generic event fields
  • For syscall events, the evt.type field is the name of the syscall
  • The evt.dir field (known as the "direction") can have two values:
    • > which indicates entry (the call is invoked) → e.g., > setuid(UID uid)
    • < which indicates exit (the call has returned) → e.g., < setuid(ERRNO res)
process Additional information about the process and thread executing the syscall event
user Information about the user executing the specific event
fd Every syscall that has a file descriptor in its arguments has these fields set with information related to the file
container
  • Container information
  • If the event is not happening inside a container, both id and name will be set to host
k8s Kubernetes related context

Manipulation

Action Description
Appending
  • Rules/Macros/Lists can be extended with append
    1. Write the element using the same name
    2. Add the append key set to true
    3. Add the appended items (list) or conditions (macro or rule)
  • Main use cases:
    • Tune noisy rules that generate false positives
    • Add additional items to lists or macros that are custom to your environment
  • Where appended:
    • When appending lists, items are added to the end of the list
    • When appending rules or macros, the additional text is appended to the condition field
Override macros
  • Some of the default rules are a placeholder which exist to be customized
  • To customize these rules, write to your local rules file a macro with the same name
Enable/Disable default rules
  • By using Falco parameters:
    • -D <substring>: Disable any rules with names having the specified substring
    • -T <tag>: Disable any rules that contains the specified tag
    • -t <tag>: Only run those rules that contain the specified tag
    • To specify in Helm charts: --set "extraArgs={-D <substring>}"
  • By using existing macros:
    • Most of the default rules offer some kind of consider_* macros, which are already part of the rule conditions
    • These macros are usually set to (never_true) or (always_true)
  • By using custom rules definitions:
    • Sometimes consider_* is not available
    • An alternative is to use the append rule properties
# There is a rule in falco_rules.yaml
- list: sensitive_directory_names
  items: [/, /etc, /etc/, /root, /root/]

# Add the following list definition to the my_rules.yaml file
- list: sensitive_directory_names
  append: true
  items: [/mnt]
# Original macro in falco_rules.yaml:
- macro: user_trusted_containers
  condition: (never_true)

# Overriding macro in falco_rules.local.yaml:
- macro: user_trusted_containers
  condition: (container.image startswith sysdig/agent)
  • Enable via existing macro:
    # Disable the default rule "Packet socket created in container"
    - macro: consider_packet_socket_communication
    condition: (always_true)
    
  • Enable via custom rule definitions:
    - rule: User mgmt binaries
    append: true
    condition: and (never_true)
    

Exceptions

There are multiple ways to deal with noisy alerts:

Method Description
Disabling one or more default rules (not recommended) Disabling the entire rule reduces the detection scope of Falco
Modifying a default rule condition Concatenate one or more conditions to the default rule to exclude a particular process, file descriptor or any other pair of field-value that identifies the exception
Defining rule exceptions If a particular rule requires tuning, an exception can be defined
- rule: <the_name_of_the_rule>
  desc: (...)
  condition: (...)
  output: (...)
  tags: (...)
  exceptions:
  - name: <name_of_the_exception>
    fields: [proc.name, fd.name]
    comps: [=, in]
    values:
      - [my_bin_A, [my_file_A]]
      - [custom_bin_B, [allowed_file_for_B, other_B_file]]

Exception that removes alerts for the Run shell untrusted rule when:

  • proc.pname = apache2
  • proc.cmdline = bash -c ls /root > /tmp/pmt
customRules:
  custom_rules_from_default: |-
    (...)
    - rule: Run shell untrusted
    (...)
      exceptions:
      - name: app1_shell_runner
        fields: [proc.pname, proc.cmdline]
        comps: [=, =]
        values:
        - [apache2, bash -c ls /root > /tmp/pmt]
    (...)

Tip

  • Fields, comps and values are grouped in tuples
  • They are joined/matched following the order in which they are defined.
  • Example:
    • The exception below considers proc.name = process_A and fd.name in file_A
    • The first field and the first value are compared using the first operator
    • The second field, with the second operator and value/values
- rule:
  (...)
  exceptions:
  - name: example_exception
    fields: [proc.name, fd.name]
    comps: [=, in]
    values:
      - [process_A, file_A]
      - [process_B, [file_B, file_C]]

Falco Alerts

  • When a Falco Rule is violated, an alert is triggered
  • Alerts can be sent to multiple supported channels
Channel Description
Standard Output
  • Default
  • Errors are printed to stdout/syslog
File Output
  • Alerts are written to file
  • Example:
    file_output:
        enabled: true
        keep_alive: true
        filename: ./events.txt
Program output
  • Redirects Falco alerts to the standard input of a desired command
  • Example (mail):
    program_output:
        enabled: true
        keep_alive: true
        program: mail -s "Falco Notification" [email protected]
  • Example (Elasticsearch):
    program_output:
        enabled: true
        keep_alive: true
        program: "curl -s -H 'Content-Type: application/json' -d @- -X POST http://localhost:9200/falco/_doc"
HTTP endpoint
  • Send Falco alerts to an HTTP[s] URL
  • Example:
    http_output:
        enabled: true
        url: http://some.url/some/path/
gRPC client
  • Not enabled by default
  • The only officially supported service using this output alternative is falco-exporter (for Prometheus)

Falco Response Engine

Falco is a Detection Engine, but it can be upgraded to a Response Engine with the help of Falco Sidekick.

Component Description
falcosidekick
  • A aemon for connecting Falco with multiple tools and protocols
  • It takes a stream of Falco events and forward them to different outputs (e.g., chats, log systems, SIEM, etc.)
  • It can also be used as a Response Engine to define mitigation actions to threats detected by Falco
  • Falco Sidekick needs to be connected to another tool that leads to the desired action. Some approaches:
sample-event-generator
  • Can generate sample activities for both syscall and k8s audit related rules
  • Installation:
    helm upgrade --namespace falco \
        --set fakeEventGenerator.enabled=true \
        --reuse-values falco falcosecurity/falco
  • The tool provides a command to run either some or all sample events:
    event-generator run [regexp]
ActivityDescriptionCall
System CallPerforms a variety of suspect actions that are detected by the default Falco rulesetevent-generator run syscall --loop
Kubernetes AuditingGenerates activity that matches the k8s audit event rulesetevent-generator run k8saudit --loop --namespace <namespace>
falco-exporter
  • Prometheus Metrics Exporter for Falco output events

Resources

General

Link Description
Sysdig Official Docs
Sysdig Falco 101 All you need to learn to get started with Falco
Sysdig Falco Hands-on Labs Learn and practice with Falco in your Browser
Sysdig Getting started with runtime security and Falco How to get started with Falco
Analyze Kubernetes Audit logs using Falco Detect intrusions that happened in your Kubernetes cluster through audit logs using Falco

Rulesets

Link Description
Using Falco to monitor outbound traffic for Pods in Kubernetes Walkthrough on how to use Falco to monitor the network traffic of your Pods