Skip to content

Commit

Permalink
Merge upstream changes
Browse files Browse the repository at this point in the history 
* Add .caseless subfield to process.name & process.executable (elastic#2341)

Adds a subfield to the process.name and process.executable fields to improve the compatibility of data sources like System, Sysmon, etc., with our Elastic Defend data, which enables us to handle language limitations in KQL more effectively.

* Revert "Add .caseless subfield to process.name & process.executable" (elastic#2350)

This reverts commit 7815b3f from elastic#2341.

This is being reverted due to storage concerns. The goal will be to advance the native querying capabilities (ES|QL, KQL) of the Elastic stack such that this extra normalized multi-field is not necessary. In the meantime, localized overrides of the ECS field definition will be used to add the additional multi-field where needed. The downside of localized overrides are that it creates inconsistency across usages of the this field.

* [RFC] Apple Platform specific fields (elastic#2338)

Adds RFS stage 0

---------

Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* Add renovate.json (elastic#2352)

Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* Update template fields (elastic#2354)

Update some templated fields that were missed before merging the RFC

* Pin dependencies (elastic#2355)

Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* Update dependency PyYAML to v6.0.2 (elastic#2356)

Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* Update dependency gitpython to v3.1.43 (elastic#2358)

Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* Update dependency yamllint to v1.35.1 (elastic#2361)

Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* Update stale PR message (elastic#2369)

Add a friendlier stale PR message, based from the
[Beats stale message](https://github.com/elastic/beats/blob/main/.github/stale.yml#L63-L74).

This will hopefully also prompt contributors to respond, so we'll be better able to track PRs
people are still interested in contributing.

* Update actions/checkout action to v4 (elastic#2362)

Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* Update actions/github-script action to v7 (elastic#2363)

Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* Update actions/setup-python action to v5 (elastic#2364)

Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* Update actions/stale action to v9 (elastic#2365)

Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* Update dependency mock to v5 (elastic#2367)

Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* Update dependency ubuntu to v22 (elastic#2368)

Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* Update dependency autopep8 to v1.7.0 (elastic#2359)

Update dependency autopep8 to v1.7.0

---------

Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* Update dependency autopep8 to v2 (elastic#2366)

* Update dependency autopep8 to v2

---------

Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* add license header (elastic#2377)

* Update actions/setup-python digest to f677139 (elastic#2374)

Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* [RFC] Stage 0: Introducing new field in rule namespace (elastic#2330)

* Update 0000-rfc-template.md

Updating the temaplate for RFC Stage 0 for adding 2 new rule fields: rule.tags and rule.remediation

* Update 0000-rfc-template.md

Incorporating review comments.

* Renaming the template file with recommended name

* Resolving conflicts

* Removing Tag Field

* Resolving comments from @trisch-me

* Moving file to rfcs/text folder as per @trisch-me comment. using next number in series.

* I saw number 44 was used in a recent RFC, using next number in series

---------

Co-authored-by: Eric Beahan <eric.beahan@elastic.co>
Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co>

* [RFC] Stage 2: Adding Apple Platform specific fields (elastic#2370)

Updating the RFC and moving it to stage two.

* code blocks specified language yaml (elastic#2380)

Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* trim trailing whitespace in schema (elastic#2379)

Co-authored-by: Michael Wolf <michael.wolf@elastic.co>

* [RFC] Stage 0: Introducing new fields in ECS vulnerability field set (elastic#2331)

* RFC to add new fields in ECS vulnerability field set

RFC to add new fields in ECS vulnerability field set

* Moving to separate file

* set title and add stage 0 PR #

* clean up fields table markdown

* Moving to (rfcs/text) and renaming file to next number in series.

* Resolving the comments from @trisch-me

* Update rfcs/text/0045-additional-vulnerability-fields.md

Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co>

* Update rfcs/text/0045-additional-vulnerability-fields.md

Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co>

* Making changed to the date format as per comments from @trisch-me

* Resolving @trisch-me comments

* Resolving latest comments

* Update rfcs/text/0045-additional-vulnerability-fields.md

Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co>

---------

Co-authored-by: Eric Beahan <eric.beahan@elastic.co>
Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co>

* Fix type in code signature (elastic#2382)

Change the type of code_signature.flags to keyword, which is what it should be. Also add a unit test that will verify all types are valid.

* Enforce yamllint in CI (elastic#2381)

Start running and enforcing yamllint checks in CI.

* Add Stage0 RFC for new fields for fileless execution on Linux (elastic#2322)

* Add support for settings

* Fix settings merging

* Restrict test workflow

* Fix merge conflicts

* Less restrictive

* Add docker files and pipeline

* Make building more restrictive

* Simplify build workflow

* Update tagging strategy

* Removing unused variable

* Kick?

* Anchors aren't supported 😭

* Fix role name

* Test branch name

* Remove extra default update (#3)

* Remove extra default update

* Fix role name

* Add support for a top-level type (#4)

* Add support for a top-level type

* Actually, don't need to be all the complicated

* Type needs to be nested within the field name (#5)

* Add documention for parameters field (#6)

* Add undocumented field argument

* Remove the PR template

---------

Co-authored-by: Jonhnathan <26856693+w0rk3r@users.noreply.github.com>
Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co>
Co-authored-by: Thijs Xhaflaire <thijsxhaflaire31@hotmail.com>
Co-authored-by: Alexandra Konrad <alexandra.konrad@elastic.co>
Co-authored-by: Michael Wolf <michael.wolf@elastic.co>
Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Stefan Bischof <bipolis@bipolis.org>
Co-authored-by: Smriti <152067238+smriti0321@users.noreply.github.com>
Co-authored-by: Eric Beahan <eric.beahan@elastic.co>
Co-authored-by: Michal Stanek <75310947+stanek-michal@users.noreply.github.com>
  • Loading branch information
@lksnyder0 @w0rk3r
@andrewkroh @txhaflaire @trisch-me @mjwolf @elastic-renovate-prod @stbischof @smriti0321 @ebeahan @stanek-michal
11 people authored Oct 9, 2024
1 parent f0f2288 commit 3cf25cb
Show file tree
Hide file tree
Showing 50 changed files with 2,082 additions and 91 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/docs-preview-comment.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ jobs:
doc-preview:
runs-on: ubuntu-latest
steps:
- uses: actions/github-script@v6
- uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7
name: Add doc preview links
with:
script: |
Expand Down
15 changes: 13 additions & 2 deletions .github/workflows/stale.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,21 @@ jobs:

steps:
- name: "Check PRs"
uses: actions/stale@v4
uses: actions/stale@28ca1036281a5e5922ead5184a1bbf96e5fc984e # v9
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
stale-pr-message: 'This PR is stale because it has been open for 60 days with no activity.'
stale-pr-message: |
Hi!
We just realized that we haven't looked into this PR in a while. We're
sorry!
We're labeling this PR as `Stale` to make it hit our filters and
make sure we get back to it as soon as possible. In the meantime, it'd
be extremely helpful if you could take a look at it as well and confirm its
relevance. A simple comment with a nice emoji will be enough `:+1`.
Thank you for your contribution!
stale-pr-label: 'stale'
ascending: true
days-before-pr-stale: 60
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/test.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,12 @@ on:

jobs:
tests:
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
name: Unit Tests
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4
- uses: actions/setup-python@f677139bbe7f9c59b41e40162b753c062f5d49a3 # v5
with:
python-version: '3.x'
- run: git fetch --prune --unshallow --tags
- run: make check
- run: make check yamllint
4 changes: 2 additions & 2 deletions Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ misspell:
fi
./build/misspell/bin/misspell -error README.md CONTRIBUTING.md schemas/* docs/* experimental/schemas/*

# Warn re misspell removal
# Warn re misspell removal
.PHONY: misspell_warn
misspell_warn:
@echo "Warning: due to lack of cross-platform support, misspell is no longer included in this task and may be deprecated in future\n"
Expand All @@ -110,4 +110,4 @@ build/ve/bin/activate: scripts/requirements.txt scripts/requirements-dev.txt
# Check YAML syntax (currently not enforced).
.PHONY: yamllint
yamllint: ve
build/ve/bin/yamllint schemas/*.yml
build/ve/bin/yamllint -d '{extends: default, rules: {line-length: disable}}' schemas/*.yml
70 changes: 67 additions & 3 deletions docs/fields/field-details.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -865,6 +865,24 @@ example: `true`

// ===============================================================

|
[[field-code-signature-flags]]
<<field-code-signature-flags, code_signature.flags>>

a| beta:[ This field is beta and subject to change. ]

The flags used to sign the process.

type: keyword



example: `570522385`

| extended

// ===============================================================

|
[[field-code-signature-signing-id]]
<<field-code-signature-signing-id, code_signature.signing_id>>
Expand Down Expand Up @@ -1610,7 +1628,7 @@ example: `co.uk`
[[ecs-device]]
=== Device Fields

Fields that describe a device instance and its characteristics. Data collected for applications and processes running on a (mobile) device can be enriched with these fields to describe the identity, type and other characteristics of the device.
Fields that describe a device instance and its characteristics. Data collected for applications and processes running on a (mobile) device can be enriched with these fields to describe the identity, type and other characteristics of the device.

This field group definition is based on the Device namespace of the OpenTelemetry Semantic Conventions (https://opentelemetry.io/docs/reference/specification/resource/semantic_conventions/device/).

Expand All @@ -1629,7 +1647,7 @@ beta::[ These fields are in beta and are subject to change.]
[[field-device-id]]
<<field-device-id, device.id>>

a| The unique identifier of a device. The identifier must not change across application sessions but stay fixed for an instance of a (mobile) device.
a| The unique identifier of a device. The identifier must not change across application sessions but stay fixed for an instance of a (mobile) device.

On iOS, this value must be equal to the vendor identifier (https://developer.apple.com/documentation/uikit/uidevice/1620059-identifierforvendor). On Android, this value must be equal to the Firebase Installation ID or a globally unique UUID which is persisted across sessions in your application.

Expand Down Expand Up @@ -1693,6 +1711,24 @@ example: `Samsung Galaxy S6`

// ===============================================================

|
[[field-device-serial-number]]
<<field-device-serial-number, device.serial_number>>

a| beta:[ This field is beta and subject to change. ]

The unique serial number serves as a distinct identifier for each device, aiding in inventory management and device authentication.

type: keyword



example: `DJGAQS4CW5`

| core

// ===============================================================

|=====


Expand Down Expand Up @@ -4811,6 +4847,24 @@ Note that this fieldset is used for common hashes that may be computed over a ra

// ===============================================================

|
[[field-hash-cdhash]]
<<field-hash-cdhash, hash.cdhash>>

a| beta:[ This field is beta and subject to change. ]

Code directory hash, utilized to uniquely identify and authenticate the integrity of the executable code.

type: keyword



example: `3783b4052fd474dbe30676b45c329e7a6d44acd9`

| extended

// ===============================================================

|
[[field-hash-md5]]
<<field-hash-md5, hash.md5>>
Expand Down Expand Up @@ -8685,6 +8739,8 @@ The `process` fields are expected to be nested at:

* `process.previous`

* `process.responsible`

* `process.session_leader`

* `process.session_leader.parent`
Expand Down Expand Up @@ -8839,6 +8895,14 @@ Note: this reuse should contain an array of process field set objects.
// ===============================================================


| `process.responsible.*`
| <<ecs-process,process>>| beta:[ This field is beta and subject to change.]

Responsible process in macOS tracks the originating process of an app, key for understanding permissions and hierarchy.

// ===============================================================


| `process.saved_group.*`
| <<ecs-group,group>>
| The saved group (sgid).
Expand Down Expand Up @@ -9142,7 +9206,7 @@ Note: this field should contain an array of values.
[[ecs-risk]]
=== Risk information Fields

Fields for describing risk score and risk level of entities such as hosts and users. These fields are not allowed to be nested under `event.*`. Please continue to use `event.risk_score` and `event.risk_score_norm` for event risk.
Fields for describing risk score and risk level of entities such as hosts and users. These fields are not allowed to be nested under `event.*`. Please continue to use `event.risk_score` and `event.risk_score_norm` for event risk.

beta::[ These fields are in beta and are subject to change.]

Expand Down
Loading

0 comments on commit 3cf25cb

Please sign in to comment.