Skip to content

Add cache_path: option to RemoteInput.open - #29

Merged
kou merged 3 commits into
red-data-tools:mainfrom
tikkss:add-input-cache-path
Oct 10, 2026
Merged

kou merged 3 commits into
red-data-tools:mainfrom
tikkss:add-input-cache-path

Conversation

@tikkss

@tikkss tikkss commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

GitHub: GH-5

cache_path: overrides the cache location computed from the URL. The
first entry is the cache ID and the rest is path in the cache:

RemoteInput.open("https://example.com/sample/data.csv",
                 cache_path: "example/sub-directory/data.csv") do |input|
  # Cached at:
  # <system cache directory>/red-remote-input/example/sub-directory/data.csv
  # "example" is cache ID and "sub-directory/data.csv" is path in the cache
end

cache_path must be <cache ID>/<path in cache>. The following paths
are raised with ArgumentError:

  • An absolute path
  • A path that includes . or ..
  • A path that doesn't have path in the cache such as data.csv

cache_path is used as is. Characters aren't replaced unlike the cache
ID computed from the URL.

Extract the cache location logic into RemoteInput::CacheEntry in
preparation. For internal use only.

I'll open a separated pull request not to change the real system cache
in tests.

tikkss added 2 commits October 2, 2026 06:53
GitHub: red-data-toolsGH-5

This is a preparation for the `cache_path:` option that overrides the
cache location. It does not change behavior.
GitHub: red-data-toolsGH-5

`cache_path:` overrides the cache location computed from the URL. The
first entry is the cache ID and the rest is path in the cache:

```ruby
RemoteInput.open("https://example.com/sample/data.csv",
                 cache_path: "example/sub-directory/data.csv") do |input|
  # Cached at:
  # <system cache directory>/red-remote-input/example/sub-directory/data.csv
end
```
Comment thread lib/remote_input/cache-entry.rb Outdated
Co-authored-by: Sutou Kouhei <kou@clear-code.com>
@tikkss

tikkss commented Oct 9, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for your review!

I've replaced from_url/from_path with CacheEntry.new.
It accepts URI or Pathname and raises ArgumentError for other objects.

Could you review it again?

@kou
kou merged commit 650a9b9 into red-data-tools:main Oct 10, 2026
9 checks passed
@kou

kou commented Oct 10, 2026

Copy link
Copy Markdown
Member

Thanks.

@tikkss
tikkss deleted the add-input-cache-path branch October 10, 2026 21:17
@tikkss

tikkss commented Oct 10, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for merging!

I'll open a separated pull request not to change the real system cache in tests.

I've opened a separated pull request: GH-30

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants