Class RemoteDatasourceDownloader

java.lang.Object
de.xima.fc.datasource.RemoteDatasourceDownloader
public class RemoteDatasourceDownloader extends Object
A utility class for downloading data from a remote datasource via HTTP GET requests. You must always provide a URL, and may customize the HTTP GET executor, status code checks, and data validity checks.

The default behavior is to allow only HTTP status code 200 (OK) and to check if the downloaded data is a valid CSV file.

Since:
8.4.0

  • Field Details

    • LOG

      protected static final org.slf4j.Logger LOG

  • Constructor Details

    • RemoteDatasourceDownloader

      public RemoteDatasourceDownloader(String url)
      Creates a new remote data source downloader with the specified URL.
      Parameters:
      url - The URL to download data from. Must not be null or empty.
      Throws:
      IllegalArgumentException - if the URL is null or empty

  • Method Details

    • allowStatusCodes

      public RemoteDatasourceDownloader allowStatusCodes(Set<Integer> allowedStatusCodes)
      Sets the allowed HTTP status codes for the download. If the response status code is not one of the these codes, the download will fail with a DatasourceDownloadException.
      Parameters:
      allowedStatusCodes - the set of allowed HTTP status codes
      Returns:
      This instance for method chaining

    • allowStatusCodes

      public RemoteDatasourceDownloader allowStatusCodes(IntPredicate allowedStatusCodes)
      Sets the allowed HTTP status codes for the download. If the response status code is not one of the these codes, the download will fail with a DatasourceDownloadException.
      Parameters:
      allowedStatusCodes - the set of allowed HTTP status codes
      Returns:
      This instance for method chaining

    • download

      public String download() throws DatasourceDownloadException
      Downloads the data from the specified URL. This method will throw a DatasourceDownloadException if the HTTP status code is not allowed (initially only 200), if the data does not pass the check function (initially only checking if content is CSV like) or an unexpected IOException occurs.
      Returns:
      the downloaded data as a String
      Throws:
      DatasourceDownloadException - if the download fails or the data is invalid

    • httpExecutor

      public RemoteDatasourceDownloader httpExecutor(RemoteDatasourceDownloader.IHttpExecutor httpGet)
      Sets the HTTP executor to use for downloading the data. This allows for custom implementations, such as using different HTTP clients or mocking the requests in tests.
      Parameters:
      httpGet - The HTTP executor to use.
      Returns:
      This instance for method chaining
      Throws:
      IllegalArgumentException - if the provided executor is null

    • validityChecker

      public RemoteDatasourceDownloader validityChecker(Predicate<byte[]> validityChecker)
      Sets the validity checker for the downloaded data. This checker is a function that takes the raw bytes and checks if the data is valid according to the specific requirements of the datasource. It must return true when the data is valid, and false if the data is invalid. If the data is invalid, the download method will throw a DatasourceDownloadException.
      Parameters:
      validityChecker - The predicate to check the validity of the downloaded data.

    • createCsvValidityChecker

      public static Predicate<byte[]> createCsvValidityChecker(CSVFormatWrapper formatWrapper)
      Creates a validity checker that checks whether the downloaded binary data is a valid UTF-8 encoded CSV file. Uses the given format wrapper to parse the CSV data.
      Parameters:
      formatWrapper - The CSV format wrapper to use for parsing the data
      Returns:
      A predicate that checks if the data is a valid CSV file according to the specified format.