What's Changed

• [Performance Analysis] print stderr if exists by @lichao127 in #8600
• libs: Update googletest by @Smjert in #8604
• Fix parsing of Windows shortcut (.lnk) files in file table by @zwass in #8601
• Fix Prefetch table for Windows 11 by @zwass in #8615
• libs: libarchive: 3.6.2 -> 3.7.9 by @LeSuisse in #8605
• Fix hardware UUID caching by @sgress454 in #8616
• Add detection for ARM CPUs when running in x86 emulation by @dantecatalfamo in #8572
• Reduce log noise for hash table by @lucasmrod in #8626
• Fix SQL example syntax in SQL introduction docs by @piotrgiedziun in #8620
• Added jetbrains_plugins table by @ksykulev in #8623
• Add recent_files table on Windows by @zwass in #8603

New Contributors

• @piotrgiedziun made their first contribution in #8620

Full Changelog: 5.17.0...5.18.1

5.17.0

Git Commits

What's Changed

• Add CHANGELOG.md entry for 5.16.0 by @lucasmrod in #8548
• Add symlink_target_path to files tables by @DocEmmetBrown in #8502
• cve: Ignore libarchive CVE-2024-26256 by @Smjert in #8546
• Fixes in windows helpers by @zwass in #8549
• Align ES functions with documented macOS versions by @SilverPlate3 in #8338
• Fix include path in logger-plugins.md by @zwass in #8550
• Fix integration test name in Windows build instructions by @zwass in #8552
• Fix event expiration to prevent losing events by @zwass in #8535
• Update shell_history table to include ash by @jbeley in #8568
• Fix dicker container table disk/write metrics, compares "op" values with ignore case by @Kislaci90 in #8566
• Escape service binary path in manage-osqueryd.ps1 by @smithclay in #8569
• Update docker_container_stats table to include memory_inactive_file and memory_total_inactive_file by @kfnorbi in #8577
• Add auto_update and app_name column to homebrew_packages table by @DocEmmetBrown in #8520
• Add support for scheduled queries to run at startup by @Micah-Kolide in #8554
• Boost 1.87 compatibility by @carlsmedstad in #8533
• Pin macos python versions in CI to fix mismatch between builder and test runner by @scottvanta in #8559
• cve: Ignore util-linux CVE-2024-28085 by @Smjert in #8579
• build(deps): bump jinja2 from 3.1.5 to 3.1.6 by @dependabot in #8563
• Fix SMC reading values by @sgress454 in #8583
• Fixes network metrics by @Kislaci90 in #8567
• Implement yara_events table for Windows by @zwass in #8580
• Fix flaky mdfind test in CI by @zwass in #8589
• libs: openssl: 3.2.1 -> 3.4.1 by @LeSuisse in #8586
• Add support for DEB822-style apt sources by @dantecatalfamo in #8556
• Add support for msix packages by @ksykulev in #8585
• Implement dns_lookup_events table on Windows by @zwass in #8553
• Added UpgradeCode to programs table by @ksykulev in #8587
• libs: expat bump from 2.6.0 to 2.7.1 by @LeSuisse in #8595
• Update ubuntu runners to 22.04 by @zwass in #8592
• Refactor ETW helpers for unicode support by @zwass in #8596
• Fix/startup items parsing by @AndreaMarangoni in #8536
• Filter the Win32_Processor query to only required fields by @jaymzjulian in #8598

New Contributors

• @DocEmmetBrown made their first contribution in #8502
• @jbeley made their first contribution in #8568
• @Kislaci90 made their first contribution in #8566
• @smithclay made their first contribution in #8569
• @kfnorbi made their first contribution in #8577
• @scottvanta made their first contribution in #8559
• @LeSuisse made their first contribution in #8586
• @dantecatalfamo made their first contribution in #8556
• @jaymzjulian made their first contribution in #8598

Full Changelog: 5.16.0...5.17.0

5.16.0

Git Commits

Representing commits from 7 contributors! Thank you all.

Table Changes

• Fix the python_paths table to skip unnecessary code paths when filtering by directory (#8544)
• Added python packages in user directories on python_packages (#8504)
• Added RHEL paths for python_packages table (#8529)
• Buffer error logs in deb_packages table (#8540)
• Fix wifi_status to correctly gather network_name on MacOS 14+ (#8530)
• Fix hardware model and version on Lenovo on system_info (#8534)
• Optimize rpm_packages and rpm_package_files use of query context (#8537)

Bug Fixes

• Fix to only deny-list scheduled queries when watchdog is enabled (#8541)
• Switched to wmain to accept non-ascii characters from command line (#8519)

5.15.0

Git Commits

Representing commits from 17 contributors! Thank you all.

Table Changes

• Add arc path to chrome_extensions on macOS (#8473)
• Use empty columns instead of zeroes when undefined in socket_events (#8510)
• Add support for accept to macOS table socket_events (#8508)
• Add all-platform user-based optimized columns (#8496)
• Add columns to es_process_events (#8506)
• Add Darwin platform optimized miscellaneous columns (#8484)
• Add all-platform path-based optimized columns (#8497)
• Add Windows platform optimized columns (#8495)
• Add hash_executable column to signature table (#8471)
• Include VSCode Insiders extensions in vscode_extensions table (#8396)
• Add POSIX platforms optimized columns (#8494)
• Add Linux platform optimized columns (#8493)
• Add all platform process based and curl optimized columns (#8498)
• Add Darwin platform optimized system-related columns (#8483)
• Add Darwin platform optimized path columns (#8482)
• Fix incorrect SID in logged_in_users table on windows when username and domain/device name are the same (#8486)
• Update the browser_firefox table to exclude "Crash Reports" and "Pending Pings" folders (#8478)
• Move status column to extended_schema for linux socket_events (#8503)

Under the Hood improvements

• Utils: Optimize default status message constructor (#8489)

Bug Fixes

• Fix a leak in genAarch64PlatformInfo (#8462)
• Fix a leak in DiskArbitrationEventPublisher::getProperty (#8463)
• Catching generic exception in order to avoid crashing when parsing windows events logs (#8513)
• Fix leak in windows_events by using scope_guard (#8511)
• Fixed eBPF's parsing of parent pid (#8501)
• Fix IO objects refcounting (#8481)

Documentation

• Add documentation for testing macOS EndpointSecurity (#8509)
• Add double quotes in Windows installation documentation (#8492)
• Update expired Slack invite (#8488)
• Update docs to correctly define conditional_to_base64 (#8460)

Build

• build(deps): bump jinja2 from 3.1.4 to 3.1.5 (#8507)
• Remove yara schema subdirectory (#8461)
• Added chrono header file (#8512)
• Replace usage of libaudit function removed in v3.0.7 (#8401)
• Update xcode version for macos-14 from 14.3.1 to 15.4 (#8467)
• Restrict python versions differently (#8453)
• Update macOS test runner from 12 to 13 (#8459)
• Add CVEs to the ignored lists (#8458)
• Add a specific package build folder on Windows jobs (#8446)
• Update all Github actions to a version using NodeJs 20 (#8449)
• Reduce scheduled builds amount (#8457)

5.14.1

Git Commits

Representing commits from 13 contributors! Thank you all.

Windows codesigning note

Starting with Osquery 5.14, we have changed our codesigning. Henceforth our releases will be signed by an osquery specific signing key issued by Microsoft Azure.

New Features

• Add --yara_sigurl_authenticate flag (#8437)

Table Changes

• Add additional WMI data to deviceguard_status table (#8440)
• Fix linux groups table to handle larger group sets by increasing buffer size (#8387)
• Add support for Firefox addons for snap installations (#8374)
• Remove support for deprecated Safari Legacy Extensions (#8426)
• macOS 15 alf support (#8428)
• Update table alf_explicit_auths as not supported on macOS 15 (#8435)
• Update table alf_exceptions to support macOS 15 (#8434)
• Fix for windows_crashes missing information on user mode memory dumps (#8394)
• Fix: safari_extensions not returning results (#8427)
• Rename hvci_status to deviceguard_status to better reflect the data collected. (#8390)

Under the Hood improvements

• Add column optimization support to allow processing IN constraints all at once in xFilter (#8263)
• Minor improvements to the hashing logic (#8398)
• Refactor readFile (#8410)

Bug Fixes

• Fix unified_log handling of timestamp formats (#8451)
• Fixes crash with non-null-terminated values in registry enumeration (#8421)
• Fix: Check and free cert context creation in windows certificates table (#8420)
• fix: Handle strftime potential error in the time table (#8431)
• Fix crash in socket table parsing on windows (#8419)

Build

• Run tests on macos-15 (#8430)
• Update tests for unified_log table to work around slowness (#8450)
• tests: Ensure python http server is ready to serve (#8452)
• Extend timeout for test HTTP server (#8445)
• Upgrade GitHub Actions upload-artifact to v4 (#8423)
• Boost 1.86 compatibility (#8409)
• build: Cleanups and fixes for a newer clang toolchain (#8412)
• ci: Update the upload-artifact action to v4.4.0 (#8416)
• build: Silence deprecation warnings about non standard extensions on VS2022 (#8405)
• Add missing includes causing compilation error with Clang 18.1.8 (#8400)
• build(deps): bump actions/download-artifact from 2 to 4.1.7 in /.github/workflows (#8411)

5.13.1

Git Commits

Representing commits from 21 contributors! Thank you all.

Windows codesigning note

The Windows binaries and MSI package have been signed with the Fleet Device Management codesigning certificate as the osquery project is currently working on identity verification to get a new signing certificate.

Table Changes

• The Python manifest directories, .egg-info and .dist-info, contain flat file hierarchies (#8318)
• Table users on linux by default to return only users in /etc/passwd (#8342)
• Add sha256 hash to apparmor_profiles table (#8345)
• Add support for metalink and store repo config file name in yum_sources table (#8307)
• Update user_ssh_keys with additional details for OpenSSL-style keys (#8314)
• Fix table dns_resolvers dns-search bug with multiple search domains (#8329)
• Fix process_open_sockets to correctly displays family and protocol on macOS (#8315)
• Add missing SSH key types to authorized_keys that support FIDO2 authentication (#8319)

Under the Hood improvements

• Improve error message when required constraint missing (#8358)
• Add verbose logging when distributed requests fail and retry (#8321)

Bug Fixes

• Fix crash in rpm_packages table by upgrading librpm from 4.18.0 to 4.18.2 #8388
• Fix crash in linux file monitoring (related to NFS mounted directories) #8392
• Fix listDirectoriesInDirectory to check if symlinks point to directories (fixes inotify warnings flooded in logs) #8399
• Fix for Potential memory leak in class ServiceArgumentParser's Constructor (#8368)
• Fix for Crash in ServiceArgumentParser via ServiceMain (#8353)
• Fixing real precision by limiting precision to 15 digits (#8355 and #8302)
• Fix invalid memory access in curl_certificates table (#8339)
• Add pending state to ATC tables to avoid duplicate sql attaches (#8324) & revert ATC changes from (#8233) that caused a race condition and ATC table failure
• Fix crash when carve size is stored as string (#8297)

Documentation

• Updated Time Machine table documentation to require FDA (#8325)
• Update processes table spec and docs, to remove outdated column alias (#8363)
• Fill in missing column descriptions to spec for device_partitions (#8364)
• Improve explanation of required columns (#8365)
• Update package_receipts table example (#8326)
• Remove some duplicated words from code comments and strings (#8336)
• Update description for alf_explicit_auths #8371

Build

• Correct spec file name to macwin (#8311)
• Correct xz submodule url and openssl download url #8383
• Update Linux Docker image to Ubuntu 20.04 (#8369)
• Fix util-linux submodule url (#8303)
• Update macos builder to 14 and tester to 12 (#8359)
• Make fallthrough explicit in sqlite_encoding.cpp (#8361)
• Fix macOS python dependencies install step (#8308)
• Bump jinja2 from 3.1.3 to 3.1.4. (#8330)

Git Commits

This release is a hot fix. It reverts #8233, which had inadvertently broken ATC tables under some conditions.

Representing commits from 3 contributors! Thank you all.

Bug Fixes

• Revert Don't add ATC table name to registry until after sqlite DB initialization #8233 (#8334)

Build

• CI: Fix macOS python dependencies install step (#8308)

Git Commits

Representing commits from 11 contributors! Thank you all.

New Features

• New flag logger_tls_backoff_max to configure the retry backoff for TLS logger plugin (#8230)

Table Changes

• Port the battery table to Windows (#8267)
• Update homebrew_packages table to include Casks (#8276)
• Update cpu_info to include load_percentage on windows (#8275)
• Check path exists first in vscode_extensions (#8292)
• deb_packages to ignore non existent admindirs (#8288)
• Add missing path separator in Safari Extensions table generator (#8273)
• Add windows UBR to os_version table (#8265)

Under the Hood improvements

• Persist query performance stats (#8250)
• Deprecate worker_threads flag (#8278)
• Change message from warning to error when extension could not be loaded (#8260)
• Refactor macOS system profile report retrieval (#8251)
• Clear performance stats when modifying scheduled/pack query (#8239)

Bug Fixes

• Fix version collate returning incorrect value when last character is a delimiter (#8283)
• Fix a memory leak in unified_log (#8274)
• Don't add ATC table name to registry until after sqlite DB initialization (#8233)

Documentation

• Update Jinja dependency for docs (#8285)
• Remove Zercurity from fleet managers list (#8293)
• Fix missing spaces in kernel_keys column descriptions (#8289)
• Update description for amperage in battery table. (#8253)

Packs

• Fix packs to check for platform before including queries (#7461)

Build

• Downgrade sqlite to 3.42 to prevent a regression with required columns (#8295)
• cve: Remove libxml2 dependency (#8282)
• cve: Update libexpat to 2.6.0 (#8281)
• cve: Update sqlite to 3.45.0 (#8259)
• cve: Update openssl to 3.2.1 (#8262)
• ci: Use all available cores and print more stats (#8248)
• cmake: Pass the osquery python path to googletest (#8237)
• test: Fix vscodeExtensions.test_sanity test (#8236)
• cmake: Correct typo, semvar -> semver (#8234)

5.11.0

Git Commits

Representing commits from 11 contributors! Thank you all.

Table Changes

• Add new table vscode_extensions (#8150)
• Add support for additional Apple Silicon columns in secureboot table (#8215)
• Add Shortcut metadata parsing on Windows in the file table (#8143)
• Remove atom_packages table (#8181)
• Add additional chrome extensions paths (#8170) to pick up extensions for Chrome Beta, Chrome Dev, and Vivaldi.

Under the Hood improvements

• Add version collations to column definitions (#8222)
• Add support for additional collations in column definitions (#8214)
• Add version collate functions (#8168)
• Added cache and throttling for certificates, keychain_acls, and keychain_items tables (#8192). This is intended to reduce the occurrence of keychain corruption due to broken macOS APIs.
• process_open_sockets: Mark pid column as additional instead of index (#8191)

Bug Fixes

• Add stricter checks to JSON parsing (#8229)
• Fix signed/unsigned mismatch in powershell_events (#8225)
• Fix a crash in firefox_addons (#8227)
• Correct the aws_sts_region behavior (#8184)

Documentation

• Update building.md prereqs for Windows (#8216)
• Correct link to a PR in the 4.7.0 changelog (#8186)
• Call out in the CHANGELOG the format changes of the status logs decorations (#8174)
• Remove some duplicated lines from 5.8.1 changelog (#8172)
• Fix typo in table specs (#8163)
• Keychain cache and throttling documentation. (#8205)
• Changelog 5.10.2 (#8171)

Build / Dependencies

• Update libxml2 to v2.12.3 (#8223)
• Update zlib to 1.3 and ignore a CVE (#8218)
• Update openssl to 3.2.0 (#8212)
• Update nvdlib to use the latest NVD APIs (#8207)
• Fix Linux build (#8208)
• Correct job order (#8185)
• Re-enable tools_tests_testrelease (#8221)
• Enable client certificate verification in the TLS tests (#8211)
• Temporary workaround to build with XCode 15 (#8197)

5.10.2

Git Commits

This release has several updates and bugfixes. Several improvements to various tables, and their handling.

One potential breaking change, is in how the watchdog calculates CPU utilization.
Previously, this calculation was based on physical CPUs, now it is based on virtual cores. We believe this makes more sense with modern CPUs.

A second potential breaking change, is in PR #8102. In addition to allowing decorations to the top level of the status logs, this PR normalizes the decorations format to the results log. In practice, this means that the unixTime, severity and line JSON fields are now numbers instead of strings.

Representing commits from 18 contributors! Thank you all.

New Features

• Add --enable_watchdog_debug flag and improve watchdog error messages (#8070)
• Add --aws_enforce_fips to enforce AWS FIPS endpoints (#8075)
• Add new AWS valid regions (#8110)
• Implement decorations_top_level flag for status logs (#8102)

Table Changes

• Add new macOS SIP config flags (#8101)
• Added cloud_id to ycloud_instance_metadata - the vm metadata table for Yandex Cloud (#8086)
• Allow querying of kernel and filesystem drivers (#8119)
• Update es_process_file_events adding support for open events, and for only triggering on file_paths (#8114)
• Update firefox_addons to use rapidjson to parse and don't block on read (#8089)
• Update macOS es_process_events table: quote spaces in command line and environment variables (#8054)
• Update linux disk_encryption to recursively query parent crypt status (#8052)
• Add, and revert, indexing on block_devices (#8037, #8151)

Under the Hood improvements

• Add warnings when an enrollment secret cannot be found (#8082)
• Avoid blocking when reading plist files (#8099)
• Fix named virtual table create statement (#8139)
• Remove forensicReadFile (#8085)
• Substitute the TEXT macro with SQL_TEXT in table code (#8091)
• Use JSON member iterator instead of rescanning (#8122)
• core: Avoid checking if a file exists before opening (#8087)
• improvement: Avoid unnecessary string conversions (#8093)
• watchdog: Use virtual cores to calculate CPU utilization limit (#8104)

Bug Fixes

• Always lock event_index_mutex when accessing event_index map (#8077)
• Check audit return values with <= (#8125)
• Fix wifi_survey table not to crash if the ssid cannot be retrieved (#8153)
• Fix macOS EndpointSecurity FIM mute inversion for file paths (#8166)

Documentation

• Add a list of Osquery fleet managers (#7781)
• Add basic file carving documentation (#8118)
• Changelog for 5.9.1 (#8088)
• Changelog 5.10.1 (#8155)
• Fixed small doc error (#8147)
• Update Automatic Table Construction example (#8094)
• Update XCode version mentions to the proper one (#8128)
• Update the description of serial_number in connected_displays (#8113)

Build

• Fix openssl build arch for Windows ARM64 (#8134)
• Fix python test http server use SSLContext.wrap_socket() instead of deprecated ssl.wrap_socket() (#8169)
• GitHub Action to cleanup at stale ec2 runners (#8156)
• Ignore CVE-2023-30571 (#8065)
• Missing pragma/header guard for boottime.h (#8117)
• Permit cross compiling for x86_64 on Apple Silicon (#8136)
• build: update macos hosted github runner to macos-12 monterey (#8100)
• ci: Fix DistributedTests.test_run_queries_with_denylisted_query test (#8154)
• ci: Increase aarch64 available space by splitting the build (#8131)
• ci: Increase disk space on the Linux x86_64 runner (#8133)
• ci: Remove flakyness when removing unused packages on Linux (#8144)
• cve: Fix the expat product name in the libraries manifest (#8158)
• cve: Ignore dbus CVE-2023-34969 (#8126)
• cve: Ignore libcap CVE-2023-2603 (#8127)
• cve: Update expat to version 2.5.0 (#8159)
• cve: Update libmagic to 5.45 (#8142)
• cve: Update lzma to 5.4.4 (#8135)
• cve: Update openssl to 3.1.3 (#8141)
• libs: Fix openssl build on aarch64 (#8084)
• libs: Update openssl to 3.1.1 (#8081)
• libs: Update openssl to 3.1.2 (#8124)
• test: Fix leaks in inotify and rocksdb tests (#8080)