Detecting syscall abuse with macOS Endpoint Security in 2026

For ten years, "writing an EDR for macOS" mostly meant writing a kernel extension. That door closed in macOS 11 — kexts are deprecated, signed kexts need notarization, and System Extensions are the supported path. The user-space half of that path is the Endpoint Security (ES) framework: a kernel-mediated event stream that surfaces ~80 curated events to a user-space es_client_t. In 2026, ES is the only practical way to ship a modern detection product on macOS.

But ES is not a syscall tracer. It exposes a hand-picked subset of system events, often at a higher abstraction level than the underlying syscall, and there's no auto-mapping. If you're moving from a Linux EDR with kprobes everywhere to a macOS deployment, you need a translation table.

This post is that table — plus the gotchas.

The shape of an ES event

Every event arrives at your client as an es_message_t:

typedef struct {
    es_event_type_t   event_type;       // e.g. ES_EVENT_TYPE_NOTIFY_OPEN
    es_action_type_t  action_type;      // AUTH (you can block) or NOTIFY (record only)
    es_process_t     *process;          // who did it
    es_event_t        event;            // event-type-specific payload
    uint64_t          time;             // mach_continuous_time
    uint64_t          mach_time;
} es_message_t;

AUTH events block until your client responds — letting you deny operations in real time. NOTIFY events are fire-and-forget. Most syscall-level events come in both flavours.

The mapping: BSD syscalls

For the POSIX-shaped surface, the most important mappings are:

The big absences are read and write: ES does not surface them. The volume would be untenable. Defenders track interesting file handles by subscribing to OPEN and remembering the resolved vnode + fd; subsequent reads/writes can be inferred from CLOSE events or from the surrounding EXEC context.

The mapping: Mach traps

The Mach side is where ES gets really interesting, because nothing else surfaces these events:

The headline event for security work is GET_TASK — that's how you catch the precondition for every macOS code-injection technique. Subscribe to AUTH_GET_TASK, build an allowlist of legitimate debugger / profiler binaries (lldb, sample, vmmap, Xcode, Activity Monitor, Instruments), and deny everything else. False-positive rate on a typical user device is very low.

For raw Mach IPC there's no event. If you need to inspect message flow, you're back to DTrace (see Tracing macOS syscalls with dtrace on Apple Silicon) or kernel-side instrumentation.

What ES gives you that syscalls don't

ES isn't just a syscall mirror — it also surfaces high-level macOS events with no direct syscall equivalent:

• ES_EVENT_TYPE_NOTIFY_LOGIN_LOGIN / _LOGOUT — TCC-style session lifecycle.
• ES_EVENT_TYPE_*_XPC_CONNECT — XPC connection setup (the intent of an IPC channel, not the messages themselves).
• ES_EVENT_TYPE_*_BTM_LAUNCH_ITEM_ADD — Background Task Management additions (persistence detection).
• ES_EVENT_TYPE_*_SUDO — sudo invocations.
• ES_EVENT_TYPE_*_SCREENSHARING_ATTACH — screen-sharing session start.

For persistence detection, the BTM_* events are far more useful than watching for LaunchAgent writes via OPEN — BTM is where Apple consolidated launch-item registration in macOS 13+, and it surfaces every Login Item, LaunchAgent, LaunchDaemon, and Helper as a structured event.

Practical gotchas

1. You need a paid Developer ID and the Endpoint Security entitlement. Apple grants it case-by-case for security-product vendors. Without it your es_new_client() returns ES_NEW_CLIENT_RESULT_ERR_NOT_PERMITTED.

2. AUTH events have a hard deadline. Each event type has a per-event response window (most are 5 seconds). Miss it and the system auto-allows + your client is marked unresponsive. Build a worker pool; don't do synchronous network calls in the handler.

3. Subscribed-event explosion. Subscribing to NOTIFY_OPEN system-wide gets you thousands of events per second on a normal Mac. Filter by es_process_t.signing_id early and aggressively, and consider muting Apple-signed system daemons.

4. AUTH_OPEN is a perf hazard. Auth events on open will slow the whole system noticeably. Most production EDRs use NOTIFY_OPEN plus selective AUTH_OPEN only for sensitive paths (Keychain, Documents directories, browser cookie jars).

5. No syscall arguments other than what ES exposes. Want to see the flags argument to open? It's in the event payload (es_event_open_t::fflag). Want to see the prot argument to mmap? Not exposed. For anything outside the curated payload, you're back to DTrace.

A minimal client

The smallest useful skeleton:

es_client_t *client = NULL;
es_new_client_result_t r = es_new_client(&client, ^(es_client_t *c, const es_message_t *msg) {
    if (msg->event_type == ES_EVENT_TYPE_AUTH_GET_TASK) {
        // Default: allow. Build your allowlist here.
        es_respond_auth_result(c, msg, ES_AUTH_RESULT_ALLOW, false);
    }
});
if (r != ES_NEW_CLIENT_RESULT_SUCCESS) { /* handle */ }

es_event_type_t events[] = { ES_EVENT_TYPE_AUTH_GET_TASK };
es_subscribe(client, events, sizeof(events) / sizeof(events[0]));

This is a task_for_pid watchdog in 10 lines. Production tooling layers process-genealogy tracking, code-signature validation, and a TLS-shipped event channel on top.

Where to go next

• The companion post task_for_pid: macOS's most dangerous Mach trap walks through what the kernel actually does when AUTH_GET_TASK fires.
• The reference catalogue gives per-syscall version history that pairs with ES event-type stability across macOS releases.
• Tracing macOS syscalls with dtrace on Apple Silicon covers the lower-level alternative when ES doesn't surface what you need.