v6.2.1

• A crash with ICMP packets involving errant length checking was fixed. Thank
  you to Ronny Barkan from Microsoft for reporting this issue.

• When a shadow file is empty/missing during rotation, Zeek aborts with an error
  message, but if the shadow file was empty, it will still be there after the
  restart. This results in an endless restart loop. This has been corrected to
  overwrite the existing shadow file, using the default file extension and
  post processing function.

• A new function remove_exclude was added to the PacketFilter framework
  which can be used to remove a previously added exclude filter by name.

• A new option --localversion was added to the configure script. This
  option allows a caller to add custom strings to the end of the Zeek version
  reported by zeek -v.

v6.0.4

This release fixes the following bugs:

• The Mozilla CA and Google CT lists were updated to their latest versions.

• A crash with ICMP packets involving errant length checking was fixed. Thank
  you to Ronny Barkan from Microsoft for reporting this issue.

• When a shadow file is empty/missing during rotation, Zeek aborts with an error
  message, but if the shadow file was empty, it will still be there after the
  restart. This results in an endless restart loop. This has been corrected to
  overwrite the existing shadow file, using the default file extension and
  post processing function.

• A new function remove_exclude was added to the PacketFilter framework
  which can be used to remove a previously added exclude filter by name.

• A new option --localversion was added to the configure script. This
  option allows a caller to add custom strings to the end of the Zeek version
  reported by zeek -v.

• The highwayhash submodule was updated to fix a build issue for FreeBSD. Thank
  you to community member Craig Leres for reporting this.

v6.2.0

We would like to thank Anthony Verez (netantho), Bijoy Das (mute019), Jan
Grashöfer (J-Gras), Matti Bispham (mbispham), Phil Rzewski (philrz), and
xb-anssi for their contributions to this release.

Breaking Changes

• The methods Dispatcher::Lookup() and Analyzer::Lookup() in the packet_analysis
  namespace were changed to return a reference to a std::shared_ptr instead of a copy
  for performance reasons.

• Zeek's OPENSSL_INCLUDE_DIR is not automatically added to an external plugin's
  include path anymore. A plugin using OpenSSL functionality directly can use the
  following explicit entry to re-use Zeek's OPENSSL_INCLUDE_DIR:

  zeek_add_plugin(
  Namespace Name
  INCLUDE_DIRS "${OPENSSL_INCLUDE_DIR}"
  SOURCES ...
  )

• The "segment_profiling" functionality and load_sample event have been removed
  without deprecation. This functionality was unmaintained and not known to be used.

• Certain ldap.log and ldap_search.log fields have been renamed from
  plural to singular and their types changed to scalars. This maps better onto
  the expected request-response protocol used between client and server. Additionally,
  it removes the burden of working with non-scalar columns from downstream systems.

  Specifically, for ldap.log:

  • arguments: vector of string is now argument: string
  • diagnostic_messages: vector of string is now diagnostic_message: string
  • objects: vector of string is now object: string
  • opcodes: set[string] is now opcode: string
  • results: set[string] is now result: string

  For ldap_search.log, the following fields were changed:

  • base_objects: vector of string is now base_object: string
  • derefs: set[string] is now deref_aliases: string
  • diagnostic_messages: vector of string is now diagnostic_message: string
  • results: set[string] is now result: string
  • scopes: set[string] is now scope: string

  In the unlikely scenario that a request-response pair with the same message
  identifier is observed, containing different values for certain fields, new
  weirds are raised and will appear in weird.log, including the old and new
  values as well as the LDAP message identifier. The value within the LDAP logs
  will be the most recently observed one.

• BIF methods now return a ValPtr directly instead of a BifReturnVal object
  which was just a thin wrapper around ValPtr. This may cause compilation errors
  in C++ code that was calling BIF methods directly.

New Functionality

• The table type was extended to allow parallel regular expression matching
  when a table's index is a pattern. Indexing such tables yields a vector
  containing all values of matching patterns for keys of type string.

  As an example, the following snippet outputs [a, a or b], [a or b].

  global tbl: table[pattern] of string;
  tbl[/a/] = "a";
  tbl[/a|b/] = "a or b";
  tbl[/c/] = "c";
  print tbl["a"], tbl["b"];

  Depending on the patterns and input used for matching, memory growth may
  be observed over time as the underlying DFA is constructed lazily. Users are
  advised to test with realistic and adversarial input data with focus on
  memory growth. The DFA's state can be reset by removal/addition of a single
  pattern. For observability, a new bif table_pattern_matcher_stats()
  can be used to gather MatcherStats.

• Support for delaying log writes.

  The logging framework offers two new functions Log::delay() and Log::delay_finish()
  to delay a Log::write() operation. This new functionality allows delaying of
  a specific log record within the logging pipeline for a variable but bounded
  amount of time. This can be used, for example, to query and wait for additional
  information to attach to the pending record, or even change its final verdict.

  Conceptually, delaying a log record happens after the execution of the global
  Log::log_stream_policy hook for a given Log::write() and before the
  execution of filter policy hooks. Any mutation of the log record within the
  delay period will be visible to filter policy hooks. Calling Log::delay()
  is currently only allowed within the context of the Log::log_stream_policy hook
  for the active Log::write()` operation (or during the execution of post delay callbacks). While this may appear restrictive, it makes it explicit which Log::write()``
  operation is subject to the delay.

  Interactions, semantics and conflicts of this feature when writing the same
  log record multiple times to the same or different log streams need to be taken
  into consideration by script writers.

  Given this is the first iteration of this feature, feedback around usability and
  use-cases that aren't covered are more than welcome.

• A WebSocket analyzer has been added together with a new websocket.log.

  The WebSocket analyzer is instantiated when a WebSocket handshake over HTTP is
  recognized. By default, the payload of WebSocket messages is fed into Zeek's dynamic
  protocol detection framework, possibly discovering and analyzing tunneled protocols.

  The format of the log and the event semantics should be considered preliminary until
  the arrival of the next long-term-stable release (7.0).

  To disable the analyzer in case of fatal errors or unexpected resource usage,
  use the Analyzer::disabled_analyzers pattern:

  redef Analyzer::disabled_analyzers += {
  Analyzer::ANALYZER_WEBSOCKET,
  };

• The SMTP analyzer was extended to recognize and properly handle the BDAT command
  from RFC 3030. This improves visibility into the SMTP protocol when mail agents
  and servers support and use this extension.

• The event keyword in signatures was extended to support choosing a custom event
  to raise instead of signature_match(). This can be more efficient in certain
  scenarios compared to funneling every match through a single event.

  The new syntax is to put the name of the event before the string used for the
  msg argument. As an extension, it is possible to only provide an event name,
  skipping msg. In this case, the framework expects the event's parameters to
  consist of only state and data as follows:

  signature only-event {
  payload /.*root/
  event found_root
  }

  event found_root(state: signature_state, data: string) { }

  Using the msg parameter with a custom event looks as follows. The custom
  event's parameters need to align with those for ``signature_match()` event:

  signature event-with-msg {
  payload /.*root/
  event found_root_with_msg "the-message"
  }

  event found_root_with_msg(state: signature_state, msg: string, data: string) { }

  Note, the message argument can currently still be specified as a Zeek identifier
  referring to a script-level string value. If used, this is disambiguated behind
  the scenes for the first variant. Specifying msg as a Zeek identifier has
  been deprecated with the new event support and will be removed in the future.

  Note that matches for signatures with custom events will not be recorded in
  signatures.log. This log is based on the generation of signature_match()
  events.

• The QUIC analyzer has been extended to support analyzing QUIC Version 2
  INITIAL packets (RFC 9369). Additionally, prior draft and some of
  Facebook's mvfst versions are supported. Unknown QUIC versions will now be
  reported in quic.log as an entry with a U history field.

• Conditional directives (@if, @ifdef, @ifndef, @else and
  @endif) can now be placed within a record's definition to conditionally
  define or extend a record type's fields.

  type r: record {
  	c: count;
  @if ( cond )
  	d: double;
  @else
  	d: count;
  @endif
  };
  

  Note that generally you should prefer record extension in conditionally loaded
  scripts rather than using conditional directives in the original record definition.

• The 'X' code can now appear in a connection's history. It is meant to indicate
  situations where Zeek stopped analyzing traffic due to exceeding certain limits or
  when encountering unknown/unsupported protocols. Its first use is to indicate
  Tunnel::max_depth being exceeded.

• A new Intel::seen_policy hook has been introduced to allow intercepting
  and changing ``Intel::seen` behavior:

  hook Intel::seen_policy(s: Intel::Seen, found: bool)

• A new NetControl::rule_added_policy hook has been introduced to allow modification
  of NetControl rules after they have been added.

• The IP geolocation / ASN lookup features in the script layer provide better
  configurability. The file names of MaxMind databases are now configurable via
  the new mmdb_city_db, mmdb_country_db, and mmdb_asn_db constants,
  and the previously hardwired fallback search path when not using an
  mmdb_dir value is now adjustable via the mmdb_dir_fallbacks
  vector. Databases opened explicitly via the mmdb_open_location_db and
  mmdb_open_asn_db functions now behave more predictably when updated or
  removed. For details, see:
  https://docs.zeek.org/en/master/customizations.html#address-geolocation-and-as-lookups

• The zeek-config script now provides a set of --have-XXX checks for
  features optionally compiled in. Each check reports "yes"/"no" to stdout and
  exits with 0/1, respectively.

Changed Functionality

• The split_string family of functions now respect the beginning-of-line ^ and
  end-of-line $ anchors. Previously, an anchored pattern would be matched anywhere
  in the input string.

• The sub() and ``gsub()` functions now respect the beginning-of-l...

v6.1.1

This release fixes the following security issues:

• A specially-crafted series of packets containing nested MIME entities can
  cause Zeek to spend large amounts of time parsing the entities. Due to the
  possibility of receiving these packets from remote hosts, this is a DoS
  risk.The fix included adds a new option (MIME::max_depth) to the MIME parser
  that limits the depth the parser will attempt to follow the entity nesting. If
  the limit is reached an exceeded_mime_max_depth weird is generated.

This release fixes the following bugs:

• CMake correctly passes along third-party package information when building
  plugins. This ensures that, for example, the same paths to OpenSSL used in a
  Zeek build are provided to a plugin build.

• Fix a problem with the HTTP analyzer where a signature regex ending in '$'
  used to match against 'http-request-body' or 'http-reply-bdoy' will never
  succeed. Thank you to GitHub user xb-anssi for this fix.

• The DNS analyzer now understands the Ed25519 and Ed448 signature algorithms.

• The SMB::State$recent_files field was not correctly expiring entries, leading
  to unbounded state growth. This is fixed to correctly follow the &read_expire
  condition on the field. Thank you to Slack user ya-sato for reporting this.

• The &create_expire attribute is now kept valid after clearing a table. After
  switching the known scripts away from broker stores, the &create_expire value
  of the local tables/sets of the known scripts wasn't in effect due to
  Cluster::node_up() and Cluster::node_down() re-assigning these without keeping
  the &create_expire attribute intact. This broke the "log hosts every 24h"
  behavior.

• Zeek builds using the --binary-package argument and including Spicy will now
  include all necessary Spicy symbols.

v6.0.3

This release fixes the following security issues:

• A specially-crafted series of packets containing nested MIME entities can
  cause Zeek to spend large amounts of time parsing the entities. Due to the
  possibility of receiving these packets from remote hosts, this is a DoS
  risk.The fix included adds a new option (MIME::max_depth) to the MIME parser
  that limits the depth the parser will attempt to follow the entity nesting. If
  the limit is reached an exceeded_mime_max_depth weird is generated.

This release fixes the following bugs:

• CMake correctly passes along third-party package information when building
  plugins. This ensures that, for example, the same paths to OpenSSL used in a
  Zeek build are provided to a plugin build.

• Fix a problem with the HTTP analyzer where a signature regex ending in '$'
  used to match against 'http-request-body' or 'http-reply-bdoy' will never
  succeed. Thank you to GitHub user xb-anssi for this fix.

• The DNS analyzer now understands the Ed25519 and Ed448 signature algorithms.

• The SMB::State$recent_files field was not correctly expiring entries, leading
  to unbounded state growth. This is fixed to correctly follow the &read_expire
  condition on the field. Thank you to Slack user ya-sato for reporting this.

• The &create_expire attribute is now kept valid after clearing a table. After
  switching the known scripts away from broker stores, the &create_expire value
  of the local tables/sets of the known scripts wasn't in effect due to
  Cluster::node_up() and Cluster::node_down() re-assigning these without keeping
  the &create_expire attribute intact. This broke the "log hosts every 24h"
  behavior.

• Zeek builds using the --binary-package argument and including Spicy will now
  include all necessary Spicy symbols.

v6.1.0

Breaking Changes

• assert is now a reserved keyword for the new assert statement.

• The __bro_plugin__ file that gets generated as part of plugin builds was
  renamed to __zeek_plugin__. This will affect the ability for older
  versions of zkg to use the zkg unload and zkg load commands. This
  should only cause breakage for people using a version of ``zkg` that doesn't
  come bundled with Zeek (which we generally don't recommend doing).

• Zeek does not traverse into dot directories to find plugins or hlto files
  anymore. Any dot directories found below the directories specified in
  ZEEK_PLUGIN_PATH or ZEEK_SPICY_MODULE_PATH are now skipped. Dot directories
  explicitly listed in ZEEK_PLUGIN_PATH or ZEEK_SPICY_MODULE_PATH are not
  skipped.

• External plugins will fail to configure if their minimum required CMake
  version is below 3.15. This was a warning with Zeek 6.0, but has caused user
  confusion due to unhelpful error messages around the IN_LIST operator policy.

• The FindBISON, FindOpenSSL, FindPackageHandleStandardArgs, FindPackageMessage,
  and SelectLibraryConfigurations cmake files were removed from our cmake
  repository in favor of the versions that come with CMake. This should not
  cause any breakage, but it is possible in the case that someone was using
  these in a plugin.

New Functionality

• Zeek now includes the LDAP protocol analyzer from the zeek/spicy-ldap project
  (https://github.com/zeek/spicy-ldap). This analyzer is enabled by default. The
  analyzer's events and its ldap.log and ldap_search.log should be
  considered preliminary and experimental until the arrival of Zeek's next
  long-term-stable release (7.0).

  If you observe unusually high CPU consumption or other issues due to this
  analyzer being enabled by default, the easiest way to disable it is via the
  Analyzer::disabled_analyzers const as follows:

  redef Analyzer::disabled_analyzers += {
  Analyzer::ANALYZER_LDAP_UDP,
  Analyzer::ANALYZER_LDAP_TCP,
  };

  Please do report issues to us including diagnostic information in case this is
  necessary in your environment. We're also open to general feedback about the
  structure of the new logs.

• Zeek now includes the QUIC protocol analyzer from the zeek/spicy-quic project
  (https://github.com/zeek/spicy-quic). This project is a fork of Fox-IT's
  initial implementation (https://github.com/fox-ds/spicy-quic).

  As for the LDAP analyzer, the analyzer's events and the new quic.log
  should be considered preliminary and experimental until the arrival of Zeek's
  next long-term-stable release (7.0). As above, any feedback and contributions
  to this analyzer and the new log are welcome.

  The analyzer's functionality is limited to decryption of the INITIAL packets
  of QUIC version 1. If decryption of these packets is successful, the handshake
  data is forwarded to Zeek's SSL analyzer. An ssl.log entry will appear in
  ssl.log for QUIC connections. The entry in the conn.log will contain
  quic and ssl in the service field.

  To disable the analyzer in case of issues, use the following snippet:

  redef Analyzer::disabled_analyzers += {
  Analyzer::ANALYZER_QUIC,
  };

• Added a new assert statement for assertion based testing and asserting
  runtime state.

  assert <expr: bool>[, <message: string>];

  This statement comes with two hooks. First, assertion_failure() that is
  invoked for every failing assert statement. Second, assertion_result()
  which is invoked for every assert statement and its outcome. The latter allows
  to construct a summary of failing and passing assert statements. Both hooks
  receive the location and call stack for the assert statement via a
  Backtrace vector.

  A failing assert will abort execution of the current event handler similar to
  scripting errors. By default, a reporter error message is logged. Using the
  break statement within assertion_failure() or assertion_result()
  allows to suppress the default message.

• Add a new &default_insert attribute for tables. This behaves as
  &default with the addition that the default value is inserted into the
  table upon a failed lookup. Particularly for tables with nested container
  values, the &default behavior of not inserting the value can be of little
  use.

• The from_json() function now takes an optional key_func argument to
  normalize JSON object key names. This can be useful if the keys in a JSON
  object are not valid Zeek identifiers or reserved keywords.

• Module names are now included in global_ids(). Their key in the returned
  table is prefixed with "module " and their value will have the type_name
  field set to "module".

• Identifiers in the global scope can now be referenced and defined from within
  modules by prefixing their names with ::. Previously, these required an
  explicit GLOBAL:: prefix to be used. Using GLOBAL:: has been
  deprecated.

• The as keyword now supports casting between set and vector values
  with the same element type. Converting set values with multiple index
  values is not supported. We plan to extend the use of the as keyword to
  support more type conversions in the future.

• Added new packet analyzer to handle PCAP files DLT_PPP link type.

• Fixed appending of any to vector of any.

• The ModBus analyzer's function support was expanded, with new handling of the
  Encapsulation Interface Transport (function 28) And Diagnostics (function 8)
  functions. This adds new modbus_encap_interface_transport_{request,response}
  and modbus_diagnostics_{request,response} events.

• The ModBus file record read and write events now provide the full data from
  the request and response messages as part of the event data.

• The full PDU length was added to the ModBusHeader record type passed with
  all of the ModBus events.

Changed Functionality

• A connection's value is now updated in-place when its directionality is
  flipped due to Zeek's heuristics (for example, SYN/SYN-ACK reversal or
  protocol specific approaches). Previously, a connection's value was discarded
  when flipped, including any values set in a new_connection() handler. A
  new connection_flipped() event is added to allow updating custom state in
  script-land.

• Loading policy/frameworks/notice/community-id.zeek now also automatically
  community ID logging. In the past, loading the script had no effect unless
  policy/protocols/conn/community-id-logging.zeek was loaded before. This
  was fairly unusual and hard to debug behavior.

• Connections to broadcast addresses are not flipped based on
  likely_server_ports anymore. Previously, broadcast packets originating
  from a likely server port resulted in 255.255.255.255 being the originator in
  conn.log.

• When too many HTTP requests are pending, Zeek will now log them at once and
  reset request/response correlation instead of running into unbounded state
  growth. This behavior is configurable via a new option
  HTTP::max_pending_requests. The default is 100.

• Fix deferred initialization of nested records containing non-const &default
  attributes.

• Parameter lists for functions, events and hooks now use commas instead of
  semicolons in error messages or when printing such functions.

• The IO buffer size used for PCAP file reading is now always 128kb. This new
  default can be changed via Pcap::bufsize_offline_bytes.

• The input framework now provides better information in error messages when
  encountering missing non-optional field while loading data.

• The SSL analyzer will now parse a configurable maximum of 10 SSL Alerts per
  SSL message. For TLS 1.3, the maximum is implicitly 1 as defined by RFC 8446.
  If there are more alerts, a new weird "SSL_excessive_alerts_in_record" is raised.
  For non-TLS 1.3, the maximum can be redefined via SSL::max_alerts_per_record.

• The ssl_history field in the ssl.log is now capped at a configurable
  limit of 100 characters prevent unbounded growth. The limit can be changed
  via the option SSL::max_ssl_history_length. When reached, a new weird
  named "SSL_max_ssl_history_length_reached" is raised.

Deprecated Functionality

• Accessing globals with GLOBAL::name has been deprecated and will be
  removed with Zeek 7.1. Use ::name instead.

• The original trigger::Trigger constructor has been deprecated and will be
  removed with Zeek 7.1. Use the new alternative constructor (per
  src/Trigger.h) instead, including replacing any use of new ... with
  make_intrusive<...>. The new constructor differs only in the placement of
  the timeout parameter, and in that - unlike the original - it always
  returns a valid pointer, which must be Unref()'d after construction, either
  explicitly (if using new) or implicitly (if using
  make_intrusive<...>).

v6.0.2

This release fixes the following security issues:

• A specially-crafted SSL packet could cause Zeek to leak memory and potentially
  crash. Due to the possibility of receiving these packets from remote hosts,
  this is a DoS risk. The fix included adds additional memory cleanup to the
  x509 file analyzer.

• A specially-crafted series of FTP packets could cause Zeek to log entries for
  requests that have already been completed, using resources unnecessarily and
  potentially causing Zeek to lose other traffic. Due to the possibility of
  receiving these packets from remote hosts, this is a DoS risk. The fix
  included changes the way that we track the pending FTP commands, avoiding
  possibly reusing the same value for subsequent commands.

• A specially-crafted series of SSL packets could cause Zeek to output a very
  large number of unnecessary alerts for the same record. Due to the possibility
  of receiving these packets from remote hosts, this is a DoS risk. The fix
  included adds a new option SSL::max_alerts_per_record that caps the number of
  alerts that can be generated for an SSL record. For TLS 1.3 this is capped at
  1 as defined in RFC 8446. For non-TLS 1.3 it is a configurable value. A
  SSL_excessive_alerts_in_record weird will be raised if the cap is exceeded.

• A specially-crafted series of SSL packets could cause Zeek to generate very
  long ssl_history fields in the ssl.log, potentially using a large amount of
  memory due to unbounded state growth. Due to the possibility of receiving
  these packets from remote hosts, this is a DoS risk. The fix included adds a
  new option SSL::max_ssl_history_length that caps this to 100 characters by
  default. A SSL_max_ssl_history_length_reached weird will be raised if the cap
  is exceeded.

• A specially-crafted IEEE802.11 packet could cause Zeek to overflow memory and
  potentially crash. Due to the possibility of receiving these packets from
  remote hosts, this is a DoS risk. The fix included adds additional bounds
  checking to the IEEE802.11 packet analyzer.

This release fixes the following bugs:

• Fixed Spicy type names from causing collisions with existing Zeek types.

• On some systems with low values for the maximum number of file descriptors, it
  was possible to run into crashes when doing DNS lookups if all of the file
  descriptors were used. This is now avoided with better checking for the number
  of available file descriptors before trying a lookup. Thank you to Zeek Slack
  user h-mikami for reporting this issue.

• Tables backed by a Broker backend now correctly support deletion if they have
  complex index types. Zeek previously reported an error when trying to delete
  elements from these tables.

• A significant performance issue with Zeek's supervisor code was fixed,
  revolving around the re-initialization of the Event Manager object used to
  track events. Thank you to Jan Grashoefer for reporting this issue.

• The MaxMind DB code now cleans up after itself, resolving a memory leak with
  the loaded database files.

• The ZeekJS submodule was updated to version 0.9.6, bringing fixes for
  zeek.invoke and zeek.event crashes, garbage collection, and an issue where
  Zeek may stop executing events from ZeekJS.

v6.0.1

This release fixes the following security issues:

• File extraction limits were not correctly enforced for files containing large
  amounts of missing bytes. Crafting files with large amounts of missing bytes
  in them could cause Zeek to spend a long time processing data, allocate a lot
  of main memory, and write a lot of data to disk. Due to the possibility of
  receiving these packets from remote hosts, this is a DoS risk. The fix
  included makes Zeek correctly enforce file size limits. We also added a new
  option (default_limit_includes_missing) which allows to customize the behavior
  when encountering large amounts of missed bytes in file. This issue was
  discovered by Luca Cigarini.

• Sessions are sometimes not cleaned up completely within Zeek during shutdown,
  potentially causing a crash when using the -B dpd flag for debug logging. This
  is low priority because it only happens at shutdown and only when using that
  flag. The fix included is to reorder shutdown to cleanup all the sessions
  prior to tearing down the analyzers.

• A specially-crafted HTTP packet can cause Zeek’s filename extraction code to
  take a long time to process the data. Due to the possibility of receiving
  these packets from remote hosts, this is a DoS risk. The fix included adjusts
  the regular expression used in the extract_filename_from_content_disposition()
  script function to more-correctly match the data.

• A specially-crafted series of FTP packets made up of a CWD request followed by
  a large amount of ERPT requests may cause Zeek to spend a long time logging
  the commands. Due to the possibility of receiving these packets from remote
  hosts, this is a DoS risk. The fix included is to prevent logging of pending
  commands for FTP packets.

• A specially-crafted VLAN packet can cause Zeek to overflow memory and
  potentially crash. Due to the possibility of receiving these packets from
  remote hosts, this is a DoS risk. The fix included is to add some additional
  length checking to the VLAN analyzer.

This release fixes the following bugs:

• Fixed a base64 decoding issue with the authorization field of HTTP request
  headers that was sometimes causing Zeek to output error messages. Thank you
  to GitHub user @progmboy for reporting and providing a fix for this issue.

• Ensure that Zeek builds use the internal version of Spicy instead of external
  installations, unless specifically configured for that mode.

• Support was added for switch fields when exporting Spicy types to Zeek.

• A number of fixes were added to protect against potential unbounded state
  growth with the SMB and DCE-RPC analyzers. SMB close requests will properly
  tear down an related DCE-RPC analyzers. A new SMB::max_dce_rpc_analyzers
  script variable was added that allows finer control of how many DCE-RPC
  analyzers are allowed to be created per SMB connection. Thanks to Zeek Slack
  user Takomi Sugawara for reporting this issue.

• Fixed a regression in the UDP and TCP analyzers that was causing more data
  than necessary to be forwarded to the next analyzer in the chain. Thanks to
  Zeek Slack user Hiep Long Tan for reporting this issue.

• A connection's value is now updated in-place when its directionality is
  flipped due to Zeek's heuristics (for example, SYN/SYN-ACK reversal or
  protocol specific approaches). Previously, a connection's value was discarded
  when flipped, including any values set in a new_connection() handler. A
  new connection_flipped() event is added to allow updating custom state in
  script-land.

• Fixed undefined symbols being reported from Spicy when building some of the
  binary packages for Zeek.

• Loading policy/frameworks/notice/community-id.zeek now also automatically
  community ID logging. In the past, loading the script had no effect unless
  policy/protocols/conn/community-id-logging.zeek was loaded before. This
  was fairly unusual and hard to debug behavior.

• Spicy no longer registers an extra port for every port registered in a
  plugin's .evt file.

• Timeouts in DNS resolution no longer cause uncontrolled memory growth.

• Fix check to skip DNS hostname lookups for notices that are not delivered via
  email in policy/frameworks/notice/extend-email/hostnames. Due to that
  policy script being loaded in the Zeek's default local.zeek, this
  previously caused unneeded DNS lookups for every generated notice instead of
  just those delivered via email.

v5.0.10

This release fixes the following security issues:

• File extraction limits were not correctly enforced for files containing large
  amounts of missing bytes. Crafting files with large amounts of missing bytes
  in them could cause Zeek to spend a long time processing data, allocate a lot
  of main memory, and write a lot of data to disk. Due to the possibility of
  receiving these packets from remote hosts, this is a DoS risk. The fix
  included makes Zeek correctly enforce file size limits. We also added a new
  option (default_limit_includes_missing) which allows to customize the behavior
  when encountering large amounts of missed bytes in file. This issue was
  discovered by Luca Cigarini.

• Sessions are sometimes not cleaned up completely within Zeek during shutdown,
  potentially causing a crash when using the -B dpd flag for debug logging. This
  is low priority because it only happens at shutdown and only when using that
  flag. The fix included is to reorder shutdown to cleanup all the sessions
  prior to tearing down the analyzers.

• A specially-crafted HTTP packet can cause Zeek’s filename extraction code to
  take a long time to process the data. Due to the possibility of receiving
  these packets from remote hosts, this is a DoS risk. The fix included adjusts
  the regular expression used in the extract_filename_from_content_disposition()
  script function to more-correctly match the data.

• A specially-crafted series of FTP packets made up of a CWD request followed by
  a large amount of ERPT requests may cause Zeek to spend a long time logging
  the commands. Due to the possibility of receiving these packets from remote
  hosts, this is a DoS risk. The fix included is to prevent logging of pending
  commands for FTP packets.

• A specially-crafted series of SMB packets containing DCE-RPC traffic may cause
  Zeek to use a large amount of memory and potentially crash. Due to the
  possibility of receiving these packets from remote hosts, this is a DoS
  risk. The fix included adds additional cleanup for DCE-RPC analyzers when
  closing SMB connections and limits on the number of DCE-RPC analyzers that can
  be created as part of an SMB connection.

This release fixes the following bugs:

• Fixed the DPD signature for the SOCKS analyzer to actually match the format
  for those packets.

• Timeouts in DNS resolution no longer cause uncontrolled memory growth.

• Fix check to skip DNS hostname lookups for notices that are not delivered via
  email in policy/frameworks/notice/extend-email/hostnames. Due to that
  policy script being loaded in the Zeek's default local.zeek, this
  previously caused unneeded DNS lookups for every generated notice instead of
  just those delivered via email.

v6.0.0

Breaking Changes

• Zeek now treats private address space (i.e., non-routable IP address ranges)
  as local by default, matching the intuition of many users that e.g. a
  192.168/16 IP address should show up as local in the logs. To do this, Zeek
  automatically adds Site::private_address_space to Site::local_nets at
  startup. Subsequent runtime updates to Site::private_address_space
  propagate to Site::local_nets, while updates to the latter don't affect
  the former.

  You're free to define Site::local_nets as before and do not need to update
  your configurations. If you added standard private address space to
  Site::local_nets in the past, you no longer need to do so. This also
  applies to zeekctl's networks.cfg file.

  The new global Boolean Site::private_address_space_is_local, true by
  default, controls the behavior. A redef to false brings back Zeek's prior
  behavior of considering private address space an unrelated concept, which will
  come in handy for example when working with tests that compare results against
  log baselines that have not yet been updated.

• Telemetry centralization and Prometheus exposition is not enabled by default
  anymore. Previously, the manager node would open port 9911/tcp by default and
  import all metrics from other nodes. For large clusters, the current implementation
  introduces significant processing overhead on the manager even if the Prometheus
  functionality is not used. While inconvenient, disable this functionality
  (assumed to be used by few as of now) by default to preserve resources.

  The script to enable centralization and the Prometheus endpoint is now
  located in the policy/ folder. Re-enable the old functionality with:

  @load frameworks/telemetry/prometheus

  You may experiment with increasing Broker::metrics_export_interval
  (default 1s) to reduce the extra overhead and communication at the expense
  of stale metrics.

• Custom source tarballs require a repo-info.json file.

  Note, should you be using official Zeek release tarballs only, or build
  Zeek solely from git checkouts, this does not affect you.

  However, if you're building your own Zeek source tarballs, it is now required
  that a repo-info.json file exists at the top-level. The dist target was
  extended to add this file and official Zeek release source tarballs will
  contain it going forward.

  The following command can be used to produce repo-info.json:

  python3 ./ci/collect-repo-info.py --only-git > ../path/to/tarballdir/repo-info.json

  This is required to support the new -V / --build-info option that
  provides information about git submodules and included plugins used during
  the build. The ci/collect-repo-info.py tool runs at ./configure time
  and either collects the required information from a git clone (when git is
  installed), or otherwise uses the content of a file named repo-info.json.

  If you see opportunities to extend repo-info.json with further information,
  please get in touch.

• Plugin authors should raise the minimum required CMake version to 3.15 to
  ensure compatibility with new CMake scaffolding included in this
  release. Older versions will trigger a warning at configuration time and,
  depending on the functionality included in the plugin, may trigger subsequent
  errors during configuration or build.

• Zeek container images are not pushed to the zeekurity organization anymore.
  Please switch to using the zeek/zeek image on DockerHub, or the images
  published to public.ecr.aws/zeek/zeek.

• The IRC_Data analyzer declaration has been moved to protocols/irc/IRC.h.

• The error message returned when using bro_init, bro_done, and
  bro_script_loaded events is now removed. removed. Usage of these events
  has returned that error during script parsing for a few years, and time has
  come to finally remove it.

New Functionality

• Zeek now features experimental JavaScript support:

  /* hello.js */
  zeek.on('zeek_init', () => {
  console.log('Hello, Zeek!');
  });

  $ zeek ./hello.js
  Hello, Zeek!

  When building Zeek on a system that features a recent (16.13+) version of the
  libnode package with development headers, Zeek automatically includes the
  externally-maintained ZeekJS plugin (https://github.com/corelight/zeekjs) as a
  builtin plugin. This allows Zeek to load and execute JavaScript code located
  in .js or .cjs files. When no such files are passed to Zeek, the
  JavaScript engine and Node.js environment aren't initialized and there is no
  runtime impact.

  The Linux distributions Fedora 37 & 38, Ubuntu 22.10, and the upcoming Debian
  12 release provide suitable packages. On other platforms, Node.js can be built
  from source with the --shared option.

  To disable this functionality, pass --disable-javascript to configure.

• Zeek now comes with Spicy support built in, meaning it can now
  leverage any analyzers written in Spicy out of the box. While the
  interface layer connecting Zeek and Spicy used to be implemented
  through an external Zeek plugin, that code has now moved into the
  Zeek code base itself. We also added infrastructure to Zeek that
  enables its built-in standard analyzers to use Spicy instead of
  Binpac. As initial (simple) examples, Zeek's Syslog and Finger
  analyzers are now implemented in Spicy. While their legacy versions
  remain available as fallbacks for now in case Spicy gets explicitly
  disabled at build time, their use is deprecated and their code won't
  be maintained any further. (Some of these Spicy updates were part of
  Zeek 5.2 already, but hadn't been included in its NEWS section.)

• Zeek events now hold network timestamps. For scheduled events, the timestamp
  represents the network time for which the event was scheduled for, otherwise
  it is the network time at event creation. A new bif current_event_time()
  allows to retrieve the current event's network timestamp within the script-layer.

  When Zeek sends events via Broker to other nodes in a cluster, an event's network
  timestamp is attached to the Broker messages. On a receiving Zeek node executing a
  handler for a remote event, current_event_time() returns the network time of
  the sending node at the time the event was created.

  The Broker level implementation allows to exchange arbitrary event metadata, but
  Zeek's script and C++ APIs currently only expose network timestamp functionality.

• A new bif from_json() can be used to parse JSON strings into records.

  type A: record { a: addr; };
  local p = from_json({"a": "192.168.0.1"}", A);
  if ( p$valid )
  print (p$v as A)

  Implicit conversion from JSON to Zeek types is implemented for bool, int, count,
  real, interval (number as seconds) and time (number as unix timestamp), port
  (strings in "80/tcp" notation), patterns, addr, subnet, enum, sets, vectors
  and records similar to the rules of the input framework. Optional or default
  record fields are allowed to be missing or null in the input.

• Zeek now provides native "Community ID" support with a new bif called
  community_id_v1(). Two policy scripts protocols/conn/community-id-logging
  and frameworks/notice/community-id extend the respective logs with a
  community_id field the same way as the external zeek-community-id plugin
  provides. A main difference to the external hash_conn() bif is that the
  community_id_v1() takes a conn_id record instead of a connection.

  Loading the new policy scripts and using the external zeek-community-id
  plugin at the same time is unsupported.

• ZeekControl is now multi-logger aware. When multiple logger nodes are configured
  in ZeekControl's node.cfg, by default the log archival logic adds a logger's name
  as suffix to the rotated file name:

  stats.11:18:57-11:19:00-logger-1.log.gz
  stats.11:18:57-11:19:00-logger-2.log.gz

  Previously, in a multi-logger setup, individual logger processes would overwrite
  each other's log files during rotation, causing data loss.

  For setups with a single logger, there's no change in behavior. The naming
  of the final logs can be customized by providing an alternative
  make-archive-name script and using the new ZEEK_ARG_LOG_SUFFIX
  environment variable.

• A supervisor controlled Zeek cluster is now multi-logger aware. This avoids
  loggers overwriting each other's log files within a single log-queue directory.
  By default, a logger's name is appended to the rotated logs by zeek-archiver.

• Introduce a new command-line option -V / --build-info. It produces
  verbose output in JSON format about the repository state and any included
  plugins.

• The X.509 certificate parser now exposes the signature type that is given inside
  the signed portion of the certificate.

• The SSL parser now parses the CertificateRequest handshake message. There is a new
  ssl_certificate_request event and a new parse_distinguished_name function.
  We also added the protocols/ssl/certificate-request-info policy script, that
  adds some additional information to ssl.log.

• Add logging metrics for streams (zeek-log-stream-writes) and writers
  (zeek-log-writer-writes-total).

• Add networking metrics via the telemetry framework. These are enabled
  when the misc/stats script is loaded.

  zeek-net-dropped-packets
  zeek-net-link-packets
  zeek-net-received-bytes
  zeek-net-packet-lag-seconds
  zeek-net-received-packets-total

  Except for lag, metrics originate from the get_net_stats() bif and are
  updated through the Telemetry::sync() hook every 15 seconds by default.

• The DNS analyzer now parses...