diff --git a/dev/.documenter-siteinfo.json b/dev/.documenter-siteinfo.json index e1551b3..abbcfd9 100644 --- a/dev/.documenter-siteinfo.json +++ b/dev/.documenter-siteinfo.json @@ -1 +1 @@ -{"documenter":{"julia_version":"1.6.7","generation_timestamp":"2023-10-10T14:11:00","documenter_version":"1.1.0"}} \ No newline at end of file +{"documenter":{"julia_version":"1.6.7","generation_timestamp":"2023-10-10T14:12:37","documenter_version":"1.1.0"}} \ No newline at end of file diff --git a/dev/index.html b/dev/index.html index 7bd681c..88db939 100644 --- a/dev/index.html +++ b/dev/index.html @@ -58,6 +58,6 @@ res = curl_execute(curl) -responses = [p.userdata for p in curl.pool] # userdata contains response data, status code and error messagesource

Globals

CurlHTTP.DEFAULT_USER_AGENTConstant

Default user agent to use if not otherwise specified. This allows an application to set the user agent string at init time rather than at constructor time.

Use CurlHTTP.setDefaultUserAgent() to set it. Set it to nothing to unset it.

source

Exported Types

CurlHTTP.CurlHandleType

Abstract type representing all types of Curl Handles. Currently CurlEasy and CurlMulti.

source
CurlHTTP.CurlEasyType

Wrapper around a curl_easy handle. This is what we get from calling curl_easy_init.

Most curl_easy_* functions will work on a CurlEasy object without any other changes.

Summary

struct CurlEasy <: CurlHandle

Fields

handle::Ptr : A C pointer to the curl_easy handle

headers::Ptr : A C pointer to the list of headers passed on to curl. We hold onto this to make sure we can free allocated memory when required.

uuid::UUID : A unique identifier for this curl handle. Used internally to identify a handle within a pool.

userdata::Dict : A dictionary of user specified data. You may add anything you want to this and it will be passed along with the curl handle to all functions. This dictionary will also be populated by several convenience functions to add the :http_status, :errormessage, and response header (:databuffer). All data added by internal code will use Symbol keys.

Constructors

CurlEasy(curl::Ptr) : Create a CurlEasy wrapper around an existing LibCURL curl handle.

CurlEasy(; url::String, method::HTTPMethod, verbose::Bool, certpath::String, keypath::String, cacertpath::String, useragent::String|Nothing) : Create a new curl object with default settings and wrap it. The default settings are:

  • FOLLOWLOCATION
  • SSL_VERIFYPEER
  • SSL_VERIFYHOST
  • SSL_VERSION (highest possible up to TLS 1.3)
  • HTTP_VERSION (H2 over TLS or HTTP/1.1)
  • TCP_FASTOPEN disabled
  • TCP_KEEPALIVE
  • ACCEPT_ENCODING best supported
  • TRANSFER_ENCODING
  • DNSCACHETIMEOUT disabled

Additionally the following options are set based on passed in parameters:

  • POST if method is POST
  • HTTPGET if method is GET
  • NOBODY if method is HEAD
  • CUSTOMREQUEST if method is HEAD, DELETE, or OPTIONS
  • VERBOSE if verbose is true
  • SSLCERT if certpath is set
  • SSLKEY if certpath and keypath are set
  • CAINFO defaults to LibCURL.cacert but can be overridden with cacertpath
  • URL if url is set
  • USERAGENT if useragent is set to something other than nothing.
source
CurlHTTP.CurlMultiType

Wrapper around a curl_multi handle. This is what we get from calling curl_multi_init

Summary

struct CurlMulti <: CurlHandle

Fields

handle::Ptr : A C pointer to the curl_multi handle

pool::CurlEasy[] : An array of CurlEasy handles that are added to this CurlMulti handle. These can be added via the constructor or via a call to curl_multi_add_handle, and may be removed via a call to curl_multi_remove_handle.

Constructors

CurlMulti() : Default constructor that calls curl_multi_init and sets up an empty pool

CurlMulti(::CurlEasy[]) : Constructor that accepts a Vector of CurlEasy objects, creates a curl_multi handle, and adds the easy handles to it.

source

Internal Types

CurlHTTP.HTTPMethodType

HTTP Methods recognized by CurlHTTP. Current values are:

  • GET: Make a GET request
  • POST: Upload data using POST
  • HEAD: Make a HEAD request and specify no response body
  • DELETE: Make a DELETE request
  • PUT: Currently not supported
  • OPTIONS: Make an OPTIONS request
source
CurlHTTP.ChannelMarkersType

Internal markers for the data channel

source

Exported Methods

CurlHTTP.curl_add_headersMethod

Add a vector of headers to the curl object

source
CurlHTTP.curl_cleanupMethod

Cleanup the CurlHandle automatically determining what needs to be done for curl_easy vs curl_multi handles. In general, this will be called automatically when the CurlHandle gets garbage collected.

source
CurlHTTP.curl_error_to_stringMethod
curl_error_to_string(::Vector{UInt8}) → String

Convert curl's error message stored as a NULL terminated sequence of bytes into a Julia String

source
CurlHTTP.curl_executeFunction
curl_execute(data_handler::Function, ::CurlEasy, ::String, ::Vector{String}; url::String) → (CURLCode, Int64, String)
-curl_execute(::CurlEasy, ::String, Vector{String}; url::String, data_handler::Function, header_handler::Function)  → (CURLCode, Int64, String)

Execute a CurlEasy handle optionally passing in a requestBody (for POSTs), any HTTP request headers, a request URL, and handlers for response data and headers.

In its first form this method accepts the data_handler as the first argument allowing you to use curl_execute(curl) do data ... end to handle the data. In this case, response headers are ignored.

In its second form, both data and header handlers are passed in as keyword arguments. If not specified, then default handlers are set up that write to userdata[:databuffer] and userdata[:responseHeaders] respectively. You may explicitly set the handler to nothing to avoid handling data or headers. This can have a small improvement in memory utilization.

source
CurlHTTP.curl_executeMethod
curl_execute(::CurlMulti) → CURLMcode

Executes all pending CurlEasy attached to the CurlMulti handle and returns a CURLMcode indicating success or failure.

In most cases, this function should return CURLM_OK even if there were failures in individual transfers. Each CurlEasy handle will have userdata[:http_status] set and userdata[:errormessage] will be set in case of an error.

This function will print errors or warnings to the Logger for unexpected states. File a bug if you see any of these.

source
CurlHTTP.curl_performMethod

Run either curl_easy_perform or curl_multi_perform depending on the type of handle passed in.

source
CurlHTTP.curl_response_statusMethod
curl_response_status(::CurlEasy) → Int64

Get the HTTP status code of the most recent response from the CurlEasy object.

source
CurlHTTP.curl_setup_requestFunction

Prepare a CurlEasy object for making a request.

  • Adds the requestBody and a corresponding Content-Length
  • Adds headers
  • If data_channel or header_channel are set, then sets up a default WRITE/HEADER callback that writes to that Channel
  • If url is set, sets the request URL
source
CurlHTTP.curl_url_escapeMethod
curl_url_escape(::CurlEasy, ::String) → String
-curl_url_escape(::String) → String

Use curl to do URL escaping

source

Methods extended from LibCURL

LibCURL.curl_easy_cleanupMethod

Cleanup everything created by the CurlEasy constructor. See the upstream docs for more details.

source
LibCURL.curl_easy_duphandleMethod

Clone a CurlEasy handle. See the upstream docs for more details.

source
LibCURL.curl_easy_escapeMethod

URL escape a Julia string using a CurlEasy handle to make it safe for use as a URL. See the upstream docs

The return value is a Julia string with memory owned by Julia, so there's no risk of leaking memory.

source
LibCURL.curl_easy_performMethod

Perform a CurlEasy transfer synchronously. See the upstream docs for more details.

source
LibCURL.curl_easy_setoptMethod

Set options for the CurlEasy handle. See the upstream docs for all possible options, and links to documentation for each option.

source
LibCURL.curl_multi_add_handleMethod

Adds a CurlEasy handle to the CurlMulti pool. See the upstream docs

source
LibCURL.curl_multi_cleanupMethod

Cleanup everything created by the CurlMulti constructor. See the upstream docs for more details.

source
LibCURL.curl_multi_performMethod

Perform all CurlEasy transfers attached to a CurlMulti handle asynchronously. See the upstream docs for more details.

source
LibCURL.curl_multi_remove_handleMethod

Remove a CurlEasy handle from the CurlMulti pool. See the upstream docs. Pass in either the CurlEasy handle or its CurlHandle.uuid.

source

Internal Methods

CurlHTTP.curl_cb_preambleMethod

Run common housekeeping tasks required by a curl callback function.

This function should be called from a curl WRITE or HEADER callback function. It does the following:

  1. Calculate the number of bytes read
  2. Copy bytes into a Vector{UInt8}
  3. Convert any non-null userdata parameter to a julia type

It then returns a tuple of these three values.

source
CurlHTTP.curl_debug_cbMethod

[Internal] curl debug callback to log informational text, header data, and SSL data transferred over the network. This will only run if curl is configured in verbose mode.

source
CurlHTTP.curl_header_cbMethod

Default header callback that puts the current header as a crlf terminate String onto a Channel passed in via curl_easy_setopt(CURLOPT_HEADERDATA).

This callback is called by curl when header data is available to be read and is set up in curl_setup_request

source
CurlHTTP.curl_setup_request_responseFunction

Setup the request object and response handlers in preparation to execute a request.

When using the CurlEasy interface, this method is called internally by curl_execute, however when using the CurlMulti interface, it is necessary to call this on every CurlEasy handle added to the CurlMulti handle.

This method allows you to set up your own response data and header handlers that receive streamed data. If you do not pass in a handler, default handlers will be set up that write binary data as bytes (Vector{UInt8}) to curl.userdata[:databuffer] and an array of String response headers (Vector{String}) to curl.userdata[:responseHeaders].

Arguments

curl::CurlEasy : The CurlEasy handle to operate on

requestBody::String : Any request body text that should be passed on to the server. Typically used for POST requests. Leave this as an empty String to skip. This is passed as-is to curl_setup_request.

headers::Vector{String} = String[] : Any request headers that should be passed on to the server as part of the request. Headers SHOULD be of the form key: value. Consult RFC 2616 section 4.2 for more details on HTTP request headers.

Keyword Arguments

data_handler::Union{Function, Nothing} = <default> : A function to handle any response Body data. This function should accept a single argument of type Vector{UInt8}. Its return value will be ignored. If not specified, a default handler will be used. Set this explicitly to nothing to disable handling of HTTP response body data.

data_handler::Union{Function, Nothing} = <default> : A function to handle any response Header data. This function should accept a single argument of type String. Its return value will be ignored. If not specified, a default handler will be used. Set this explicitly to nothing to disable handling of HTTP response header data.

url::AbstractString="" : The URL to use for this request. This permanently overrides the url passed in to the CurlEasy constructor. If not specified, then the previous value of the CurlEasy's url is reused.

Returns

The CurlEasy object.

source
CurlHTTP.curl_write_cbMethod

Default write callback that puts the data stream as a Vector{UInt8} onto a Channel passed in via curl_easy_setopt(CURLOPT_WRITEDATA).

This callback is called by curl when data is available to be read and is set up in curl_setup_request

source
CurlHTTP.setDefaultUserAgentMethod

Set the default user agent string to use for all requests. Set this to nothing to disable setting the user agent string.

source
CurlHTTP.setup_response_handlerMethod

Setup a Channel for the default response handlers to write to.

source

Index

+responses = [p.userdata for p in curl.pool] # userdata contains response data, status code and error messagesource

Globals

CurlHTTP.DEFAULT_USER_AGENTConstant

Default user agent to use if not otherwise specified. This allows an application to set the user agent string at init time rather than at constructor time.

Use CurlHTTP.setDefaultUserAgent() to set it. Set it to nothing to unset it.

source

Exported Types

CurlHTTP.CurlHandleType

Abstract type representing all types of Curl Handles. Currently CurlEasy and CurlMulti.

source
CurlHTTP.CurlEasyType

Wrapper around a curl_easy handle. This is what we get from calling curl_easy_init.

Most curl_easy_* functions will work on a CurlEasy object without any other changes.

Summary

struct CurlEasy <: CurlHandle

Fields

handle::Ptr : A C pointer to the curl_easy handle

headers::Ptr : A C pointer to the list of headers passed on to curl. We hold onto this to make sure we can free allocated memory when required.

uuid::UUID : A unique identifier for this curl handle. Used internally to identify a handle within a pool.

userdata::Dict : A dictionary of user specified data. You may add anything you want to this and it will be passed along with the curl handle to all functions. This dictionary will also be populated by several convenience functions to add the :http_status, :errormessage, and response header (:databuffer). All data added by internal code will use Symbol keys.

Constructors

CurlEasy(curl::Ptr) : Create a CurlEasy wrapper around an existing LibCURL curl handle.

CurlEasy(; url::String, method::HTTPMethod, verbose::Bool, certpath::String, keypath::String, cacertpath::String, useragent::String|Nothing) : Create a new curl object with default settings and wrap it. The default settings are:

  • FOLLOWLOCATION
  • SSL_VERIFYPEER
  • SSL_VERIFYHOST
  • SSL_VERSION (highest possible up to TLS 1.3)
  • HTTP_VERSION (H2 over TLS or HTTP/1.1)
  • TCP_FASTOPEN disabled
  • TCP_KEEPALIVE
  • ACCEPT_ENCODING best supported
  • TRANSFER_ENCODING
  • DNSCACHETIMEOUT disabled

Additionally the following options are set based on passed in parameters:

  • POST if method is POST
  • HTTPGET if method is GET
  • NOBODY if method is HEAD
  • CUSTOMREQUEST if method is HEAD, DELETE, or OPTIONS
  • VERBOSE if verbose is true
  • SSLCERT if certpath is set
  • SSLKEY if certpath and keypath are set
  • CAINFO defaults to LibCURL.cacert but can be overridden with cacertpath
  • URL if url is set
  • USERAGENT if useragent is set to something other than nothing.
source
CurlHTTP.CurlMultiType

Wrapper around a curl_multi handle. This is what we get from calling curl_multi_init

Summary

struct CurlMulti <: CurlHandle

Fields

handle::Ptr : A C pointer to the curl_multi handle

pool::CurlEasy[] : An array of CurlEasy handles that are added to this CurlMulti handle. These can be added via the constructor or via a call to curl_multi_add_handle, and may be removed via a call to curl_multi_remove_handle.

Constructors

CurlMulti() : Default constructor that calls curl_multi_init and sets up an empty pool

CurlMulti(::CurlEasy[]) : Constructor that accepts a Vector of CurlEasy objects, creates a curl_multi handle, and adds the easy handles to it.

source

Internal Types

CurlHTTP.HTTPMethodType

HTTP Methods recognized by CurlHTTP. Current values are:

  • GET: Make a GET request
  • POST: Upload data using POST
  • HEAD: Make a HEAD request and specify no response body
  • DELETE: Make a DELETE request
  • PUT: Currently not supported
  • OPTIONS: Make an OPTIONS request
source
CurlHTTP.ChannelMarkersType

Internal markers for the data channel

source

Exported Methods

CurlHTTP.curl_add_headersMethod

Add a vector of headers to the curl object

source
CurlHTTP.curl_cleanupMethod

Cleanup the CurlHandle automatically determining what needs to be done for curl_easy vs curl_multi handles. In general, this will be called automatically when the CurlHandle gets garbage collected.

source
CurlHTTP.curl_error_to_stringMethod
curl_error_to_string(::Vector{UInt8}) → String

Convert curl's error message stored as a NULL terminated sequence of bytes into a Julia String

source
CurlHTTP.curl_executeFunction
curl_execute(data_handler::Function, ::CurlEasy, ::String, ::Vector{String}; url::String) → (CURLCode, Int64, String)
+curl_execute(::CurlEasy, ::String, Vector{String}; url::String, data_handler::Function, header_handler::Function)  → (CURLCode, Int64, String)

Execute a CurlEasy handle optionally passing in a requestBody (for POSTs), any HTTP request headers, a request URL, and handlers for response data and headers.

In its first form this method accepts the data_handler as the first argument allowing you to use curl_execute(curl) do data ... end to handle the data. In this case, response headers are ignored.

In its second form, both data and header handlers are passed in as keyword arguments. If not specified, then default handlers are set up that write to userdata[:databuffer] and userdata[:responseHeaders] respectively. You may explicitly set the handler to nothing to avoid handling data or headers. This can have a small improvement in memory utilization.

source
CurlHTTP.curl_executeMethod
curl_execute(::CurlMulti) → CURLMcode

Executes all pending CurlEasy attached to the CurlMulti handle and returns a CURLMcode indicating success or failure.

In most cases, this function should return CURLM_OK even if there were failures in individual transfers. Each CurlEasy handle will have userdata[:http_status] set and userdata[:errormessage] will be set in case of an error.

This function will print errors or warnings to the Logger for unexpected states. File a bug if you see any of these.

source
CurlHTTP.curl_performMethod

Run either curl_easy_perform or curl_multi_perform depending on the type of handle passed in.

source
CurlHTTP.curl_response_statusMethod
curl_response_status(::CurlEasy) → Int64

Get the HTTP status code of the most recent response from the CurlEasy object.

source
CurlHTTP.curl_setup_requestFunction

Prepare a CurlEasy object for making a request.

  • Adds the requestBody and a corresponding Content-Length
  • Adds headers
  • If data_channel or header_channel are set, then sets up a default WRITE/HEADER callback that writes to that Channel
  • If url is set, sets the request URL
source
CurlHTTP.curl_url_escapeMethod
curl_url_escape(::CurlEasy, ::String) → String
+curl_url_escape(::String) → String

Use curl to do URL escaping

source

Methods extended from LibCURL

LibCURL.curl_easy_cleanupMethod

Cleanup everything created by the CurlEasy constructor. See the upstream docs for more details.

source
LibCURL.curl_easy_duphandleMethod

Clone a CurlEasy handle. See the upstream docs for more details.

source
LibCURL.curl_easy_escapeMethod

URL escape a Julia string using a CurlEasy handle to make it safe for use as a URL. See the upstream docs

The return value is a Julia string with memory owned by Julia, so there's no risk of leaking memory.

source
LibCURL.curl_easy_performMethod

Perform a CurlEasy transfer synchronously. See the upstream docs for more details.

source
LibCURL.curl_easy_setoptMethod

Set options for the CurlEasy handle. See the upstream docs for all possible options, and links to documentation for each option.

source
LibCURL.curl_multi_add_handleMethod

Adds a CurlEasy handle to the CurlMulti pool. See the upstream docs

source
LibCURL.curl_multi_cleanupMethod

Cleanup everything created by the CurlMulti constructor. See the upstream docs for more details.

source
LibCURL.curl_multi_performMethod

Perform all CurlEasy transfers attached to a CurlMulti handle asynchronously. See the upstream docs for more details.

source
LibCURL.curl_multi_remove_handleMethod

Remove a CurlEasy handle from the CurlMulti pool. See the upstream docs. Pass in either the CurlEasy handle or its CurlHandle.uuid.

source

Internal Methods

CurlHTTP.curl_cb_preambleMethod

Run common housekeeping tasks required by a curl callback function.

This function should be called from a curl WRITE or HEADER callback function. It does the following:

  1. Calculate the number of bytes read
  2. Copy bytes into a Vector{UInt8}
  3. Convert any non-null userdata parameter to a julia type

It then returns a tuple of these three values.

source
CurlHTTP.curl_debug_cbMethod

[Internal] curl debug callback to log informational text, header data, and SSL data transferred over the network. This will only run if curl is configured in verbose mode.

source
CurlHTTP.curl_header_cbMethod

Default header callback that puts the current header as a crlf terminate String onto a Channel passed in via curl_easy_setopt(CURLOPT_HEADERDATA).

This callback is called by curl when header data is available to be read and is set up in curl_setup_request

source
CurlHTTP.curl_setup_request_responseFunction

Setup the request object and response handlers in preparation to execute a request.

When using the CurlEasy interface, this method is called internally by curl_execute, however when using the CurlMulti interface, it is necessary to call this on every CurlEasy handle added to the CurlMulti handle.

This method allows you to set up your own response data and header handlers that receive streamed data. If you do not pass in a handler, default handlers will be set up that write binary data as bytes (Vector{UInt8}) to curl.userdata[:databuffer] and an array of String response headers (Vector{String}) to curl.userdata[:responseHeaders].

Arguments

curl::CurlEasy : The CurlEasy handle to operate on

requestBody::String : Any request body text that should be passed on to the server. Typically used for POST requests. Leave this as an empty String to skip. This is passed as-is to curl_setup_request.

headers::Vector{String} = String[] : Any request headers that should be passed on to the server as part of the request. Headers SHOULD be of the form key: value. Consult RFC 2616 section 4.2 for more details on HTTP request headers.

Keyword Arguments

data_handler::Union{Function, Nothing} = <default> : A function to handle any response Body data. This function should accept a single argument of type Vector{UInt8}. Its return value will be ignored. If not specified, a default handler will be used. Set this explicitly to nothing to disable handling of HTTP response body data.

data_handler::Union{Function, Nothing} = <default> : A function to handle any response Header data. This function should accept a single argument of type String. Its return value will be ignored. If not specified, a default handler will be used. Set this explicitly to nothing to disable handling of HTTP response header data.

url::AbstractString="" : The URL to use for this request. This permanently overrides the url passed in to the CurlEasy constructor. If not specified, then the previous value of the CurlEasy's url is reused.

Returns

The CurlEasy object.

source
CurlHTTP.curl_write_cbMethod

Default write callback that puts the data stream as a Vector{UInt8} onto a Channel passed in via curl_easy_setopt(CURLOPT_WRITEDATA).

This callback is called by curl when data is available to be read and is set up in curl_setup_request

source
CurlHTTP.setDefaultUserAgentMethod

Set the default user agent string to use for all requests. Set this to nothing to disable setting the user agent string.

source
CurlHTTP.setup_response_handlerMethod

Setup a Channel for the default response handlers to write to.

source

Index