Package de.xima.fc.datasource

Class RemoteDatasourceDownloader

java.lang.Object

de.xima.fc.datasource.RemoteDatasourceDownloader

public class RemoteDatasourceDownloader
extends Object

A utility class for downloading data from a remote datasource via HTTP GET requests. You must always provide a URL, and may customize the HTTP GET executor, status code checks, and data validity checks.

The default behavior is to allow only HTTP status code 200 (OK) and to check if the downloaded data is a valid CSV file.

Since:

8.4.0

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	RemoteDatasourceDownloader.HttpGetResponse	POJO for the response of an HTTP GET request.
static interface	RemoteDatasourceDownloader.IHttpExecutor	Interface for executing HTTP GET requests.

Field Summary

Fields

Modifier and Type	Field	Description
protected static org.slf4j.Logger	LOG

Constructor Summary

Constructors

Constructor	Description
RemoteDatasourceDownloader(String url)	Creates a new remote data source downloader with the specified URL.

Method Summary

Modifier and Type	Method	Description
RemoteDatasourceDownloader	allowStatusCodes(IntPredicate allowedStatusCodes)	Sets the allowed HTTP status codes for the download.
RemoteDatasourceDownloader	allowStatusCodes(Set<Integer> allowedStatusCodes)	Sets the allowed HTTP status codes for the download.
static Predicate<byte[]>	createCsvValidityChecker(CSVFormatWrapper formatWrapper)	Creates a validity checker that checks whether the downloaded binary data is a valid UTF-8 encoded CSV file.
String	download()	Downloads the data from the specified URL.
RemoteDatasourceDownloader	httpExecutor(RemoteDatasourceDownloader.IHttpExecutor httpGet)	Sets the HTTP executor to use for downloading the data.
RemoteDatasourceDownloader	validityChecker(Predicate<byte[]> validityChecker)	Sets the validity checker for the downloaded data.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

protected static final org.slf4j.Logger LOG

Constructor Detail

RemoteDatasourceDownloader

public RemoteDatasourceDownloader(String url)

Creates a new remote data source downloader with the specified URL.

Parameters:

url - The URL to download data from. Must not be null or empty.

Throws:

IllegalArgumentException - if the URL is null or empty

Method Detail

allowStatusCodes

public RemoteDatasourceDownloader allowStatusCodes(Set<Integer> allowedStatusCodes)

Sets the allowed HTTP status codes for the download. If the response status code is not one of the these codes, the download will fail with a DatasourceDownloadException.

Parameters:

allowedStatusCodes - the set of allowed HTTP status codes

Returns:

This instance for method chaining

allowStatusCodes

public RemoteDatasourceDownloader allowStatusCodes(IntPredicate allowedStatusCodes)

Sets the allowed HTTP status codes for the download. If the response status code is not one of the these codes, the download will fail with a DatasourceDownloadException.

Parameters:

allowedStatusCodes - the set of allowed HTTP status codes

Returns:

This instance for method chaining

download

public String download()
                throws DatasourceDownloadException

Downloads the data from the specified URL. This method will throw a DatasourceDownloadException if the HTTP status code is not allowed (initially only 200), if the data does not pass the check function (initially only checking if content is CSV like) or an unexpected IOException occurs.

Returns:

the downloaded data as a String

Throws:

DatasourceDownloadException - if the download fails or the data is invalid

httpExecutor

public RemoteDatasourceDownloader httpExecutor(RemoteDatasourceDownloader.IHttpExecutor httpGet)

Sets the HTTP executor to use for downloading the data. This allows for custom implementations, such as using different HTTP clients or mocking the requests in tests.

Parameters:

httpGet - The HTTP executor to use.

Returns:

This instance for method chaining

Throws:

IllegalArgumentException - if the provided executor is null

validityChecker

public RemoteDatasourceDownloader validityChecker(Predicate<byte[]> validityChecker)

Sets the validity checker for the downloaded data. This checker is a function that takes the raw bytes and checks if the data is valid according to the specific requirements of the datasource. It must return true when the data is valid, and false if the data is invalid. If the data is invalid, the download method will throw a DatasourceDownloadException.

Parameters:

validityChecker - The predicate to check the validity of the downloaded data.

createCsvValidityChecker

public static Predicate<byte[]> createCsvValidityChecker(CSVFormatWrapper formatWrapper)

Creates a validity checker that checks whether the downloaded binary data is a valid UTF-8 encoded CSV file. Uses the given format wrapper to parse the CSV data.

Parameters:

formatWrapper - The CSV format wrapper to use for parsing the data

Returns:

A predicate that checks if the data is a valid CSV file according to the specified format.