Add an example to demonstrate multithreaded read_parquet pipelines

[mhaseeb123]

Description

Closes #16717. This PR adds a new example to read multiple parquet files using multiple threads.

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[GregoryKimball]

Thanks @mhaseeb123 for putting this together. After a little hacking I was able to generate a nice pipelined profile. I removed the concats and the writes and only kept the first read. I also had to add an initialization for thread_count to prevent garbage thread values.

Here is nice pipelining

From this command:
/nfs/nsight-systems-2022.5.1/bin/nsys profile -t nvtx,cuda,osrt -f true --cuda-memory-usage=true --cuda-um-cpu-page-faults=true --cuda-um-gpu-page-faults=true --gpu-metrics-device=4 --output=/nfs/20240924_iothreads/prof2_4 --env-var KVIKIO_COMPAT_MODE=on,KVIKIO_NTHREADS=8 ./parquet_io_multithreaded /raid/tpch/gqe-dbgen-1/lineitem/lineitem_1.snappy.parquet,/raid/tpch/gqe-dbgen-1/part/part_1.snappy.parquet,/raid/tpch/gqe-dbgen-1/orders/orders_1.snappy.parquet,/raid/tpch/gqe-dbgen-1/lineitem/lineitem_1.snappy.parquet,/raid/tpch/gqe-dbgen-1/part/part_1.snappy.parquet,/raid/tpch/gqe-dbgen-1/orders/orders_1.snappy.parquet,/raid/tpch/gqe-dbgen-1/lineitem/lineitem_1.snappy.parquet,/raid/tpch/gqe-dbgen-1/part/part_1.snappy.parquet,/raid/tpch/gqe-dbgen-1/orders/orders_1.snappy.parquet,/raid/tpch/gqe-dbgen-1/lineitem/lineitem_1.snappy.parquet,/raid/tpch/gqe-dbgen-1/part/part_1.snappy.parquet,/raid/tpch/gqe-dbgen-1/orders/orders_1.snappy.parquet,/raid/tpch/gqe-dbgen-1/lineitem/lineitem_1.snappy.parquet,/raid/tpch/gqe-dbgen-1/part/part_1.snappy.parquet,/raid/tpch/gqe-dbgen-1/orders/orders_1.snappy.parquet,/raid/tpch/gqe-dbgen-1/lineitem/lineitem_1.snappy.parquet,/raid/tpch/gqe-dbgen-1/part/part_1.snappy.parquet,/raid/tpch/gqe-dbgen-1/orders/orders_1.snappy.parquet /raid/tmp DEFAULT SNAPPY 4 YES

Based on this command, you can see why I would like a "repetitions" CLI argument 😆

[GregoryKimball]

Thanks @mhaseeb123 for the continued development and discussions around this example.

• I would also like to consider introducing an io_type CLI argument similar to the way the nvbenchmarks work. We could take the file information and copy it to pageable host buffers, pinned host buffers, or device buffers, based on the io_type requested by the runner.
• Also I'm seeing poor pipelining in the first set of read_parquet calls, as each thread launches its IO at the same time, and we end up starting compute later than if each thread completed IO one at a time. I'll open a separate issue about this topic. ([FEA] Add synchronization for IO between read_parquet calls on different threads #16936)

[mhaseeb123]

Based on this command, you can see why I would like a "repetitions" CLI argument 😆

OMG that command line is such a monstrosity. All done in the update though. Still working on the io_type

[vuule]

the final batch of nits :)

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[GregoryKimball]

@lamarrr Would you please share your review?

[bdice]

CI changes look fine to me.

[mhaseeb123]

/merge