Skip to content

Commit

Permalink
prepare 0.10.0 release
Browse files Browse the repository at this point in the history
  • Loading branch information
digizeph committed Feb 11, 2024
1 parent 571004b commit 209234a
Show file tree
Hide file tree
Showing 4 changed files with 195 additions and 64 deletions.
97 changes: 97 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,103 @@

All notable changes to this project will be documented in this file.

## v0.10.0 - 2024-02-12

Version 0.10.0 is a major release with a lot of changes and improvements.

### Highlights

#### MRT Encoding

`bgpkit-parser` now supports encoding MRT messages. The following MRT message types are supported:
- TableDumpV1
- TableDumpV2
- BGP4MP
It also supports encoding BMP messages into MRT files.

Example of writing BgpElems into a file RIB dump file:
```rust
let mut encoder = bgpkit_parser::encoder::MrtRibEncoder::new();
for elem in parser {
encoder.process_elem(&elem);
}
let mut writer = oneio::get_writer("filtered.rib.gz").unwrap();
writer.write_all(encoder.export_bytes().as_ref()).unwrap();
drop(writer);
```

Another example of writing `BgpElem` to BGP updates bytes:
```rust
let mut encoder = bgpkit_parser::encoder::MrtUpdatesEncoder::new();
let mut elem = BgpElem::default();
elem.peer_ip = IpAddr::V4("10.0.0.1".parse().unwrap());
elem.peer_asn = Asn::from(65000);
elem.prefix.prefix = "10.250.0.0/24".parse().unwrap();
encoder.process_elem(&elem);
elem.prefix.prefix = "10.251.0.0/24".parse().unwrap();
encoder.process_elem(&elem);
let bytes = encoder.export_bytes();

let mut cursor = Cursor::new(bytes.clone());
while cursor.has_remaining() {
let parsed = parse_mrt_record(&mut cursor).unwrap();
dbg!(&parsed);
}
```

See `encoder` module for more details.

#### Better developer experiences

- added several utility functions to `BgpElem`
- `.is_announcement()`: check if the BGP element is an announcement
- `.get_as_path_opt()`: get the AS path if it exists and no AS set or confederated segments
- `.get_origin_asn_opt()`: get the origin ASN if it exists
- full `serde` serialization support
- add `BgpElem` to PSV (pipe-separated values) conversion
- improved time-related filters parsing
- `ts_start` `start_ts` `ts_end` `end_ts` are all supported
- many quality of life improvements by [@jmeggitt](https://github.com/jmeggitt)
- https://github.com/bgpkit/bgpkit-parser/pull/122
- https://github.com/bgpkit/bgpkit-parser/pull/123

#### Improved testing coverage

- `bgpkit-parser` code test coverage is now at 92%.
- codecov.io coverage report is available at https://app.codecov.io/gh/bgpkit/bgpkit-parser

#### TLS backend choice

- by default, `bgpkit-parser` now uses `rustls` as the default TLS backend
- `openssl` is still supported, but it is no longer the default
- `openssl` support can be enabled by using the `native-tls` feature flag and set default features to `false`

### Added RFCs support

- [`RFC 4724: Graceful Restart Mechanism for BGP`][rfc4724]
- [`RFC 8671 Support for Adj-RIB-Out in the BGP Monitoring Protocol (BMP)`][rfc8671]
- [`RFC 9069: Support for Local RIB in the BGP Monitoring Protocol (BMP)`][rfc9069]
- the supported RFCs list is documented at https://github.com/bgpkit/bgpkit-parser?tab=readme-ov-file#rfcs-support

[rfc4724]: https://www.rfc-editor.org/rfc/rfc4724
[rfc8671]: https://www.rfc-editor.org/rfc/rfc8671
[rfc9069]: https://www.rfc-editor.org/rfc/rfc9069

### Fixes

- fixed a bug where when multiple `AsSequences` are present, only the first one is parsed
- issue: https://github.com/bgpkit/bgpkit-parser/issues/140
- fixed a bug where the parser panics when messages are truncated
- https://github.com/bgpkit/bgpkit-parser/issues/149

### Other changes

- Move pybgpkit to its own repository at https://github.com/bgpkit/pybgpkit
- CLI build feature changed from `build-binary` to `cli`
- add more ways to install compiled `bgpkit-parser` CLI
- homebrew on macOS: `brew install bgpkit/tap/bgpkit-parser`
- other platforms: `cargo binstall bgpkit-parser`

## v0.10.0-beta.3 - 2024-02-08

### Highlights
Expand Down
2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "bgpkit-parser"
version = "0.10.0-beta.3"
version = "0.10.0"
authors = ["Mingwei Zhang <[email protected]>"]
edition = "2021"
license = "MIT"
Expand Down
81 changes: 49 additions & 32 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# BGPKIT Parser

*This readme is generated from the library's doc comments using [cargo-readme](https://github.com/livioribeiro/cargo-readme). Please refer to the Rust docs website for the full documentation: [latest stable](https://docs.rs/bgpkit-parser/latest/bgpkit_parser/); [bleeding-edge](https://docs.rs/bgpkit-parser/0.10.0-beta.3/bgpkit_parser/).*
*This readme is generated from the library's doc comments using [cargo-readme](https://github.com/livioribeiro/cargo-readme). Please refer to the Rust docs website for the full documentation: [latest stable](https://docs.rs/bgpkit-parser/latest/bgpkit_parser/); [bleeding-edge](https://docs.rs/bgpkit-parser/0.10.0/bgpkit_parser/).*

[![Build](https://github.com/bgpkit/bgpkit-parser/actions/workflows/build.yml/badge.svg)](https://github.com/bgpkit/bgpkit-parser/actions/workflows/build.yml)
[![Crates.io](https://img.shields.io/crates/v/bgpkit-parser)](https://crates.io/crates/bgpkit-parser)
Expand Down Expand Up @@ -55,7 +55,7 @@ total: 255849
[BGPKIT Broker][broker-repo] library provides search API for all RouteViews and RIPE RIS MRT data files. Using the
broker's Rust API ([`bgpkit-broker`][broker-crates-io]), we can easily compile a list of MRT files that we are interested
in for any time period and any data type (`update` or `rib`). This allows users to gather information without needing to
know about locations of specific data files.
know about the locations of specific data files.

[broker-repo]: https://github.com/bgpkit/bgpkit-broker
[broker-crates-io]: https://crates.io/crates/bgpkit-broker
Expand Down Expand Up @@ -100,7 +100,7 @@ for item in broker.into_iter().take(2) {

### Filtering BGP Messages

BGPKIT Parser also has built-in [Filter] mechanism. When creating a new [`BgpkitParser`] instance,
BGPKIT Parser also has a built-in [Filter] mechanism. When creating a new [`BgpkitParser`] instance,
once can also call `add_filter` function to customize the parser to only show matching messages
when iterating through [BgpElem]s.

Expand All @@ -109,7 +109,7 @@ For all types of filters, check out the [Filter] enum documentation.
```rust
use bgpkit_parser::BgpkitParser;

/// This example shows how to parse a MRT file and filter by prefix.
/// This example shows how to parse an MRT file and filter by prefix.
env_logger::Builder::from_env(env_logger::Env::default().default_filter_or("info")).init();

log::info!("downloading updates file");
Expand Down Expand Up @@ -243,47 +243,64 @@ drop(mrt_writer);

`bgpkit-parser` is bundled with a utility commandline tool `bgpkit-parser-cli`.

### Installation

#### Install compiled binaries

You can install the compiled `bgpkit-parser` CLI binaries with the following methods:
- **Homebrew** (macOS): `brew install bgpkit/tap/bgpkit-parser`
- [**Cargo binstall**](binstall): `cargo binstall bgpkit-parser`

[binstall]: https://github.com/cargo-bins/cargo-binstall

#### From source

You can install the tool by running
```bash
cargo install bgpkit-parser --features cli
```
or checkout this repository and run
```bash
cargo install --path ./bgpkit-parser --features cli
cargo install --path . --features cli
```

### Usage

Run `bgpkit-parser --help` to see the full list of options.

```
➜ cli git:(cli) ✗ bgpkit-parser-cli 0.1.0
Mingwei Zhang <[email protected]>
bgpkit-parser-cli is a simple cli tool that allow parsing of individual MRT files
USAGE:
bgpkit-parser-cli [FLAGS] [OPTIONS] <FILE>
FLAGS:
-e, --elems-count Count BGP elems
-h, --help Prints help information
--json Output as JSON objects
--pretty Pretty-print JSON output
-r, --records-count Count MRT records
-V, --version Prints version information
OPTIONS:
-a, --as-path <as-path> Filter by AS path regex string
-m, --elem-type <elem-type> Filter by elem type: announce (a) or withdraw (w)
-T, --end-ts <end-ts> Filter by end unix timestamp inclusive
-o, --origin-asn <origin-asn> Filter by origin AS Number
-J, --peer-asn <peer-asn> Filter by peer IP ASN
-j, --peer-ip <peer-ip> Filter by peer IP address
-p, --prefix <prefix> Filter by network prefix
-t, --start-ts <start-ts> Filter by start unix timestamp inclusive
MRT/BGP/BMP data processing library
Usage: bgpkit-parser [OPTIONS] <FILE>
Arguments:
<FILE> File path to a MRT file, local or remote
Options:
-c, --cache-dir <CACHE_DIR> Set the cache directory for caching remote files. Default behavior does not enable caching
--json Output as JSON objects
--psv Output as full PSV entries with header
--pretty Pretty-print JSON output
-e, --elems-count Count BGP elems
-r, --records-count Count MRT records
-o, --origin-asn <ORIGIN_ASN> Filter by origin AS Number
-p, --prefix <PREFIX> Filter by network prefix
-s, --include-super Include super-prefix when filtering
-S, --include-sub Include sub-prefix when filtering
-j, --peer-ip <PEER_IP> Filter by peer IP address
-J, --peer-asn <PEER_ASN> Filter by peer ASN
-m, --elem-type <ELEM_TYPE> Filter by elem type: announce (a) or withdraw (w)
-t, --start-ts <START_TS> Filter by start unix timestamp inclusive
-T, --end-ts <END_TS> Filter by end unix timestamp inclusive
-a, --as-path <AS_PATH> Filter by AS path regex string
-h, --help Print help
-V, --version Print version
```

## Data Representation

There are two key data structure to understand for the parsing results: [MrtRecord] and [BgpElem].
There are two key data structures to understand for the parsing results: [MrtRecord] and [BgpElem].

### `MrtRecord`: unmodified MRT information representation

Expand Down
79 changes: 48 additions & 31 deletions src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ total: 255849
[BGPKIT Broker][broker-repo] library provides search API for all RouteViews and RIPE RIS MRT data files. Using the
broker's Rust API ([`bgpkit-broker`][broker-crates-io]), we can easily compile a list of MRT files that we are interested
in for any time period and any data type (`update` or `rib`). This allows users to gather information without needing to
know about locations of specific data files.
know about the locations of specific data files.
[broker-repo]: https://github.com/bgpkit/bgpkit-broker
[broker-crates-io]: https://crates.io/crates/bgpkit-broker
Expand Down Expand Up @@ -90,7 +90,7 @@ for item in broker.into_iter().take(2) {
## Filtering BGP Messages
BGPKIT Parser also has built-in [Filter] mechanism. When creating a new [`BgpkitParser`] instance,
BGPKIT Parser also has a built-in [Filter] mechanism. When creating a new [`BgpkitParser`] instance,
once can also call `add_filter` function to customize the parser to only show matching messages
when iterating through [BgpElem]s.
Expand All @@ -99,7 +99,7 @@ For all types of filters, check out the [Filter] enum documentation.
```no_run
use bgpkit_parser::BgpkitParser;
/// This example shows how to parse a MRT file and filter by prefix.
/// This example shows how to parse an MRT file and filter by prefix.
env_logger::Builder::from_env(env_logger::Env::default().default_filter_or("info")).init();
log::info!("downloading updates file");
Expand Down Expand Up @@ -235,47 +235,64 @@ drop(mrt_writer);
`bgpkit-parser` is bundled with a utility commandline tool `bgpkit-parser-cli`.
## Installation
### Install compiled binaries
You can install the compiled `bgpkit-parser` CLI binaries with the following methods:
- **Homebrew** (macOS): `brew install bgpkit/tap/bgpkit-parser`
- [**Cargo binstall**](binstall): `cargo binstall bgpkit-parser`
[binstall]: https://github.com/cargo-bins/cargo-binstall
### From source
You can install the tool by running
```bash
cargo install bgpkit-parser --features cli
```
or checkout this repository and run
```bash
cargo install --path ./bgpkit-parser --features cli
cargo install --path . --features cli
```
## Usage
Run `bgpkit-parser --help` to see the full list of options.
```text
➜ cli git:(cli) ✗ bgpkit-parser-cli 0.1.0
Mingwei Zhang <[email protected]>
bgpkit-parser-cli is a simple cli tool that allow parsing of individual MRT files
USAGE:
bgpkit-parser-cli [FLAGS] [OPTIONS] <FILE>
FLAGS:
-e, --elems-count Count BGP elems
-h, --help Prints help information
--json Output as JSON objects
--pretty Pretty-print JSON output
-r, --records-count Count MRT records
-V, --version Prints version information
OPTIONS:
-a, --as-path <as-path> Filter by AS path regex string
-m, --elem-type <elem-type> Filter by elem type: announce (a) or withdraw (w)
-T, --end-ts <end-ts> Filter by end unix timestamp inclusive
-o, --origin-asn <origin-asn> Filter by origin AS Number
-J, --peer-asn <peer-asn> Filter by peer IP ASN
-j, --peer-ip <peer-ip> Filter by peer IP address
-p, --prefix <prefix> Filter by network prefix
-t, --start-ts <start-ts> Filter by start unix timestamp inclusive
MRT/BGP/BMP data processing library
Usage: bgpkit-parser [OPTIONS] <FILE>
Arguments:
<FILE> File path to a MRT file, local or remote
Options:
-c, --cache-dir <CACHE_DIR> Set the cache directory for caching remote files. Default behavior does not enable caching
--json Output as JSON objects
--psv Output as full PSV entries with header
--pretty Pretty-print JSON output
-e, --elems-count Count BGP elems
-r, --records-count Count MRT records
-o, --origin-asn <ORIGIN_ASN> Filter by origin AS Number
-p, --prefix <PREFIX> Filter by network prefix
-s, --include-super Include super-prefix when filtering
-S, --include-sub Include sub-prefix when filtering
-j, --peer-ip <PEER_IP> Filter by peer IP address
-J, --peer-asn <PEER_ASN> Filter by peer ASN
-m, --elem-type <ELEM_TYPE> Filter by elem type: announce (a) or withdraw (w)
-t, --start-ts <START_TS> Filter by start unix timestamp inclusive
-T, --end-ts <END_TS> Filter by end unix timestamp inclusive
-a, --as-path <AS_PATH> Filter by AS path regex string
-h, --help Print help
-V, --version Print version
```
# Data Representation
There are two key data structure to understand for the parsing results: [MrtRecord] and [BgpElem].
There are two key data structures to understand for the parsing results: [MrtRecord] and [BgpElem].
## `MrtRecord`: unmodified MRT information representation
Expand Down

0 comments on commit 209234a

Please sign in to comment.