![]() Lexicographical order sorts items based on the values used to encode the items in computer memory. To use the dedup command on multivalue fields, the fields must match all values to be deduplicated. This behavior applies to any field with high cardinality and large size. If you search the _raw field, the text of every event in memory is retained which impacts your search performance. See Command types.Īvoid using the dedup command on the _raw field if you are searching over a large volume of data. All of the results must be collected before sorting. For example, if you specify the ![]() The dedup command is a streaming command or a dataset processing command, depending on which arguments are specified with the command. str Syntax: str() Description: Order the field values by using the lexicographic order. num Syntax: num() Description: Interpret the field values as numbers. ip Syntax: ip() Description: Interpret the field values as IP addresses. auto Syntax: auto() Description: Determine automatically how to sort the field values. Syntax: Description: The name of the field to sort. Sort field options Syntax: | auto() | str() | ip() | num() Description: The options that you can specify to sort the events. When the list of events is sorted, the top-most event, of the duplicate events in the sorted list, is retained. The determines which of the duplicate events to keep. You must specify the sort order for each field specified in the. Use the dash symbol ( - ) for descending order and the plus symbol ( + ) for ascending order. ![]() Syntax: sortby ( - | + ) Description: List of the fields to sort by and the sort order. All other duplicates are removed from the results. If you do not specify a number, only the first occurring event is kept. Syntax: Description: The dedup command retains multiple events for each combination when you specify N. Events are dropped after the first event of each particular combination. | fillnull value="MISSING" field1 field2 | dedup field1 field2 keepevents Syntax: keepevents= Description: If true, keep all events, but will remove the selected fields from events after the first event containing a particular combination of values. ![]() To keep N representative events for combinations of field values including null values, use the fillnull command to provide a non-null value for these fields. The keepempty=true argument keeps every event that does not have one or more of the fields in the field list. All events where any of the selected fields are null are dropped. Default: false keepempty Syntax: keepempty= Description: If set to true, keeps every event where one or more of the specified fields is not present (null). Optional arguments consecutive Syntax: consecutive= Description: If true, only remove events with duplicate combinations of values that are consecutive. Description: A list of field names to remove duplicate values from. Other options enable you to retain events with the duplicate fields removed, or to keep events where the fields specified do not exist in the events.Äedup Required arguments Syntax. You can sort the fields, which determines which event is retained. You can specify the number of events with duplicate values, or value combinations, to keep. For real-time searches, the first events that are received are searched, which are not necessarily the most recent events. For historical searches, the most recent events are searched first. Events returned by dedup are based on search order. With the dedup command, you can specify the number of duplicate events to keep for each value of a single field, or for each combination of values among several fields. Removes the events that contain an identical combination of values for the fields that you specify.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |