24 Nov 09:31

Xuanwo

48c48b1

v0.55.0 Latest

Latest

Rust Core Upgrade to v0.55

Public API

Timestamp types now come from `jiff`

All public metadata APIs that previously exposed chrono::DateTime<Utc> now use jiff::Timestamp. For example, Metadata::last_modified() and related setters return/accept Timestamp values (core/src/types/metadata.rs). Update downstream crates to depend on jiff if they manipulate these timestamps or convert them to other formats.

Scheme handling is string-based

OperatorInfo::scheme() now returns &'static str instead of Scheme, and Operator::via_iter accepts impl AsRef<str> (typically the services::*_SCHEME constants). Additionally, the deprecated constructors Operator::from_map and Operator::via_map have been removed. Migrate any code that relied on the enum variants or the removed constructors to the new string-based constants and from_iter/via_iter.

List APIs only support `versions`

OpList::with_version()/version() and Capability::list_with_version have been removed after a long deprecation cycle. Use with_versions()/versions() on OpList and read Capability::list_with_versions instead.

`S3Builder::security_token` removed

S3Builder no longer exposes the deprecated security_token() helper. Use session_token() exclusively when configuring temporary credentials.

KV-style services no longer pretend to support `list`

Services that never returned meaningful results for Operator::list (such as D1, FoundationDB, GridFS, Memcached, MongoDB, MySQL, Persy, PostgreSQL, Redb, Redis, SurrealDB, TiKV, etc.) now rely on the default Unsupported implementation. Those features will be implemented later.

Raw API

Deprecated KV adapters removed

The legacy opendal::raw::adapters::{kv, typed_kv} modules have been deleted. Services should directly implement Access instead of depending on the adapters. Remove the corresponding imports and shim layers from any out-of-tree services.

Bindings Python Upgrade to v0.47

Breaking change: Module exports are explicit

opendal.__init__ now only re-exports the capability, exceptions, file, layers, services, types, Operator, and AsyncOperator symbols. Imports such as:

from opendal import Metadata, Layer

no longer work. Update them to use the dedicated submodules:

from opendal.types import Metadata
from opendal.layers import Layer

The legacy helper module opendal.__base has also been removed together with _Base.

Breaking change: Capability accessors renamed

Both Operator.full_capability() and AsyncOperator.full_capability() have been renamed to capability(). Adjust your code accordingly:

-caps = op.full_capability()
+caps = op.capability()

Breaking change: Service identifiers now have typed enums

The constructors for Operator / AsyncOperator provide overloads that accept opendal.services.Scheme members. While plain strings are still accepted at runtime, type checkers (pyright/mypy) expect the new enum values. Migrate code bases that relied on importing the old Scheme enum from opendal to from opendal import services and use services.Scheme.<NAME>.

What's Changed

Added

feat(services/oss): allow users to specify endpoint addressing style by @howardlau1999 in #6504
feat(bindings/ruby): publish ruby binding by @erickguan in #6539
feat(services/s3): add support for HTTP 429 TooManyRequests for S3-compatible services by @justinjoseph89 in #6589
feat: Add Operator Registry by @Xuanwo in #6608
feat: Add from_uri support for more object storage services by @Xuanwo in #6665
feat(services/webdav): Implement write returns metadata by @yunchipang in #6660
feat: Add from_uri support for http/webdav/ftp/sftp by @Xuanwo in #6666
feat(services/swift): Implement write returns metadata by @yunchipang in #6669
feat(buffer): add chunk iterator for Buffer by @TennyZhuang in #6672
feat(services/dropbox): Implement write returns metadata by @yunchipang in #6673
feat: Add from_uri support for all services by @Xuanwo in #6668
chore(bindings/python): Add Justfile and format codebase by @chitralverma in #6677
feat: Introduce TailCutLayer by @Xuanwo in #6680
feat(services/gdrive): Implement write returns metadata by @yunchipang in #6683
feat(bindings/python): Add stubs for Exception and Capability by @chitralverma in #6690
feat(bindings/python): Add stubs for some more types by @chitralverma in #6703
feat(bindings/python)!: Add stubs for remaining types by @chitralverma in #6720
feat(bindings/python)!: Stubs fix for operator and version by @chitralverma in #6728
RFC-6707: Capability Override Layer by @Xuanwo in #6707
feat(bindings/python)!: Generate stubs for Operator overloads and Scheme by @chitralverma in #6729
feat(bindings/nodejs): add concurrent limit layer by @kingsword09 in #6739
feat(bindings/python): Enable ftp/hdfs_native/sftp for python by @chitralverma in #6745
feat: implement content-md5 for s3 by @waynexia in #6508
feat(services/compfs): implement IoVectoredBuf for Buffer by @meteorgan in #6762
feat(services/lakefs): Implement write returns metadata by @kingsword09 in #6770
Add connection_pool_max_size option to memcached and redis builders by @trxcllnt in #6759
fix:(bindings/python) Gate service-sftp on windows targets by @chitralverma in #6777
feat(binding/nodejs): add TimeoutLayer, LoggingLayer and ThrottleLayer in nodejs binding by @Kilerd in #6772

Changed

refactor: Migrate cloudflare_kv from adapter::kv to Access instead by @kingsword09 in #6340
refactor: Centralize date/time handling with Timestamp wrapper by @dami0806 in #6650
refactor: Move Config related back to config.rs instead by @Xuanwo in #6667
refactor: Remove deprecated project virtiofs by @Xuanwo in #6682
refactor(core): remove conditional compilation in PageList trait by @crwen in #6715
refactor: migrate tikv service from adapter::kv to impl Access directly by @koushiro in #6713
refactor: migrate memcached service from adapter::kv to impl Access directly by @koushiro in #6714
refactor: migrate mysql service from adapter::kv to impl Access directly by @koushiro in #6716
refactor: migrate postgresql service from adapter::kv to impl Access directly by @koushiro in #6718
refactor: migrate persy service from adapter::kv to impl Access directly by @koushiro in #6721
refactor: migrate foundationdb service from adapter::kv to impl Access directly by @koushiro in #6722
refactor: migrate surrealdb service from adapter::kv to impl Access directly by @koushiro in #6723
refactor: migrate mongodb service from adapter::kv to impl Access directly by @koushiro in #6724
refactor: migrate d1 service from adapter::kv to impl Access directly by @koushiro in #6725
refactor: migrate gridfs service from adapter::kv to impl Access directly by @koushiro in #6726
refactor: Migrate sqlite from adapter::kv to Access instead by @NoxTav in #6328
refactor: migrate sled service from adapter::kv to impl Access directly by @koushiro in #6731
refactor: migrate rocksdb service from adapter::kv to impl Access directly by @koushiro in #6732
refactor: migrate redb service from adapter::kv to impl Access directly by @koushiro in #6733
refactor: Remove deprecated kv and typed_kv adapters by @kingsword09 in #6734
refactor(hdfs): restructure HdfsBackend and introduce HdfsCore by @kingsword09 in #6736
refactor(hdfs-native): restructure HdfsNativeBackend and introduce HdfsNativeCore by @kingsword09 in #6737
refactor(services): impl Debug for all service builders by @koushiro in #6756
refactor(types)!: use string-based scheme over enum-based approach by @koushiro in #6765

Fixed

fix(services/cloudflare_kv): Use DEFAULT_SCHEME constant for C...

Contributors

trxcllnt, erickguan, and 32 other contributors

Assets 2

0 Join discussion

11 Sep 15:18

Xuanwo

v0.54.1

b93c3ae

v0.54.1

What's Changed

Added

feat: Conditional reader for cos by @pmupkin in #6424
feat: Add project layout for openval by @Xuanwo in #6427
feat: Add if-not-exists support for copy with azblob as example by @Xuanwo in #6447
feat(bindings/cpp): enhance Metadata API with comprehensive feature a… by @JackDrogon in #6445
feat: Add prefetch support for concurrent read by @Xuanwo in #6449
feat(bindings/cpp): adopt Google C++ Style Guide for API naming by @JackDrogon in #6448
feat(object_store): Implement copy, rename and return metadata after writing by @codephage2020 in #6452
feat(object_store): Implement copy_if_not_exists, put_multipart_opts and metadata in GetResult by @codephage2020 in #6468
chore(deps): add groups for nodejs by @asukaminato0721 in #6469
feat(bindings/ocaml): add more blocking api for ocaml by @asukaminato0721 in #6238
fix(bindings/cpp): Add capability to fix failed tests by @asukaminato0721 in #6488
feat(bindings/cpp): enable user-configurable features for storage services by @KinomotoMio in #6503
feat(services/oss): add support for security token for Aliyun OSS by @cavivie in #6511
feat(services/azblob): Add compatibility aliases for Apache Arrow object_store by @jackye1995 in #6527
feat(services/gcs): add configuration aliases for better Arrow object_store compatibility by @jackye1995 in #6526
feat(services/s3): add configuration aliases for better Arrow object_store compatibility by @jackye1995 in #6524
feat: allow using object_store as opendal's backend by @flaneur2020 in #6283
feat(services/opfs): Add create_dir support for OPFS by @crwen in #6542
feat(bindings/ruby): support file open options by @erickguan in #6538

Changed

refactor(website): remove unused dependencies and update node engine requirement by @kingsword09 in #6421
refacrtor: Reduce usage of scheme inside raw by @Xuanwo in #6457
refactor: Migrate etcd service to implement Access directly by @kingsword09 in #6420

Fixed

fix(bin/oli): Fix tests when TMPDIR is not /tmp by @XYenon in #6429
fix: patch futures buffer concurrent by @tisonkun in #6442
fix(services/oss): unnecessary header 'content-type' are included in the get_object request by @honsunrise in #6455
fix(services/moka): prevent the weigher from being overwritten by @koushiro in #6498
fix(services/azblob): Fix azblob base64 validation by @kenwoodjw in #6500
fix(bindings/nodejs): use value.prefetch instead of value.concurrent for prefetch option by @kingsword09 in #6502
fix(integrations/object_store): remove redundant into_iter() call by @KinomotoMio in #6509
fix(bindings/cpp): correct cxx_async namespace attribute by @kingsword09 in #6514
Fix: make cpp binding build robust by @devillove084 in #6460
fix: Do not trim trailing slash of name_node by @wForget in #6556

Docs

docs(release): fix CI workflow links for bindings by @kingsword09 in #6437
docs: Update RFC-5444 Operator From Uri by @Xuanwo in #6453
docs(core): update docs of building a storage service by @crwen in #6554

CI

ci: remove docker-compose version top-level elements by @kingsword09 in #6436
ci: Update ci_bindings_zig.yml by @tisonkun in #6444
ci: enable trust publishing for rust crates by @kingsword09 in #6454
ci(bindings/cpp): cpp test framework by @asukaminato0721 in #6296
ci: Use ubuntu-24.04-arm for java release to address glibc issues by @Xuanwo in #6486
ci: refactor nodejs ci workflow by @kingsword09 in #6491
ci: update workflow nodejs version and remove corepack by @kingsword09 in #6495
chore(ci): upgrade 1password/load-secrets-action v3 by @kingsword09 in #6517
chore: adjust CI workflow names for bindings by @erickguan in #6540
chore(ci): migrate etcd, memcached, redis, and redis-cluster images to new sources by @kingsword09 in #6435

Chore

chore(deps): bump quick-xml from 0.37.5 to 0.38.0 in /bin/oay by @dependabot[bot] in #6416
chore(deps): bump the http-serialization-utils group in /core with 6 updates by @dependabot[bot] in #6414
chore: Fix clippy for object_store by @Xuanwo in #6422
chore(deps): bump the others group across 1 directory with 17 updates by @dependabot[bot] in #6432
chore(website): upgrade prism-react-renderer to v2.4.1 by @kingsword09 in #6433
chore(deps): bump tokio from 1.46.1 to 1.47.0 in /bin/oay by @dependabot[bot] in #6466
chore(deps): bump tokio from 1.46.1 to 1.47.0 in /bin/oli by @dependabot[bot] in #6464
chore(deps): bump tokio from 1.46.1 to 1.47.0 in /bin/ofs by @dependabot[bot] in #6463
chore(deps): bump the others group in /core with 2 updates by @dependabot[bot] in #6478
chore(deps): bump indicatif from 0.17.11 to 0.18.0 in /bin/oli by @dependabot[bot] in #6474
chore(deps): bump the http-serialization-utils group in /core with 2 updates by @dependabot[bot] in #6476
chore(deps): bump tokio from 1.46.1 to 1.47.1 in /core in the async-runtime group by @dependabot[bot] in #6477
chore(bindings/nodejs): upgrade deps napi-rs to 3.0 by @kingsword09 in #6482
chore: make clippy happy again by @kingsword09 in #6493
chore(bindings/nodejs): update generated code comments style due to napi-derive indentation fix by @kingsword09 in #6501
feat: expose APIs for ErrorStatus by @tisonkun in #6513
chore(layers/fastmetrics): upgrade fastmetrics dependency to v0.4 by @koushiro in #6534
chore(deps-dev): bump crates-llms-txt from 0.0.8 to 0.1.1 in /website by @dependabot[bot] in #6548
chore(deps): bump logforth from 0.26.1 to 0.27.0 in /bin/oay by @dependabot[bot] in #6546
chore(deps): bump tempfile from 3.20.0 to 3.21.0 in /bin/oli by @dependabot[bot] in #6544
chore(deps): update datafusion requirement from 48.0.0 to 49.0.2 in /integrations/object_store by @dependabot[bot] in #6545
chore(deps): bump logforth from 0.26.1 to 0.27.0 in /bin/ofs by @dependabot[bot] in #6547
Bump version to 0.54.1 by @Xuanwo in #6558

New Contributors

@pmupkin made their first contribution in #6424
@XYenon made their first contribution in #6429
@codephage2020 made their first contribution in #6452
@honsunrise made their first contribution in #6455
@kenwoodjw made their first contribution in #6500
@KinomotoMio made their first contribution in #6503
@cavivie made their first contribution in #6511
@devillove084 made their first contribution in #6460
@jackye1995 made their first contribution in #6527
@crwen made their first contribution in #6542
@wForget made their first contribution in #6556

Full Changelog: v0.54.0...v0.54.1

Contributors

flaneur2020, erickguan, and 18 other contributors

Assets 2

5 Join discussion

17 Jul 07:06

Xuanwo

v0.54.0

a1b8109

v0.54.0

Upgrade Notes

Rust Core Public API

RFC-6189: Remove Native Blocking Support

OpenDAL v0.54 implements RFC-6189, which removes all native blocking support in favor of using block_on from async runtimes.

The following breaking changes have been made:

blocking::Operator can no longer be used within async contexts
Using blocking APIs now requires an async runtime
All Blocking* types have been moved to the opendal::blocking module

To migrate:

- use opendal::BlockingOperator;
+ use opendal::blocking::Operator;

RFC-6213: Options Based API

OpenDAL v0.54 implements RFC-6213, which introduces options-based APIs for more structured and extensible operation configuration.

New APIs added:

read_options(path, ReadOptions)
write_options(path, data, WriteOptions)
list_options(path, ListOptions)
stat_options(path, StatOptions)
delete_options(path, DeleteOptions)

Example usage:

// Read with options
let options = ReadOptions::new()
    .range(0..1024)
    .if_match("etag");
let data = op.read_options("path/to/file", options).await?;

// Write with options  
let options = WriteOptions::new()
    .content_type("text/plain")
    .cache_control("max-age=3600");
op.write_options("path/to/file", data, options).await?;

Remove `stat_has_xxx` and `list_has_xxx` APIs

All stat_has_* and list_has_* capability check APIs have been removed. Instead, check capabilities directly on the Capability struct:

- if op.info().full_capability().stat_has_content_length() {
+ if op.info().full_capability().stat.content_length {
    // ...
}

Fix `with_user_metadata` signature

The signature of with_user_metadata has been changed. Please update your code accordingly if you use this method.

Services removed due to lack of maintainer

The following services have been removed due to lack of maintainers:

atomicserver
icloud
nebula_graph

If you need these services, please consider maintaining them or use alternative services.

HttpClientLayer replaces `update_http_client`

The Operator::update_http_client() method has been replaced by HttpClientLayer:

- op.update_http_client(client);
+ op = op.layer(HttpClientLayer::new(client));

Expose `presign_xxx_options` API

New options-based presign APIs have been exposed:

let options = PresignOptions::new()
    .expire(Duration::from_secs(3600));
    
let url = op.presign_read_options("path/to/file", options).await?;

Rust Core Raw API

Remove native blocking support

All native blocking implementations have been removed from the raw API. Services and layers no longer need to implement blocking-specific methods.

Bindings Java Breaking changes

Removed services

The following services have been removed:

Chainsafe service has been removed (PR-5744) - The service has been sunset.
libsql service has been removed (PR-5616) - Dead service removal.

Batch operations removed

PR-5393 removes the batch concept from OpenDAL. All batch-related operations and capabilities have been removed.

Capability changes

write_multi_align_size capability has been removed (PR-5322)
Added new shared capability (PR-5328)

Options-based API

New options classes have been introduced for structured operation configuration:

ReadOptions - for read operations
WriteOptions - for write operations
ListOptions - for list operations
StatOptions - for stat operations

Example usage:

// Read with options
ReadOptions options = ReadOptions.builder()
    .range(0, 1024)
    .ifMatch("etag")
    .build();
byte[] data = operator.read("path/to/file", options);

// Write with options
WriteOptions options = WriteOptions.builder()
    .contentType("text/plain")
    .cacheControl("max-age=3600")
    .build();
operator.write("path/to/file", data, options);

Bindings Python

Breaking change: Native blocking API removed

OpenDAL has removed the native blocking API in the core. The Python binding's blocking API now uses an async runtime internally. This is a transparent change and should not affect most users, but:

The blocking API now requires an async runtime to be available
Performance characteristics may be slightly different
All blocking operations are now implemented as async operations running in a tokio runtime

Breaking change: Removed services

The following services have been removed due to lack of maintainers and users:

atomicserver - This service is no longer supported
icloud - This service is no longer supported
nebula_graph - This service is no longer supported

If you were using any of these services, you'll need to migrate to an alternative storage backend.

Breaking change: Chainsafe service removed

The Chainsafe service has been sunset and is no longer available.

What's Changed

Added

RFC-6213: Options API by @Xuanwo in #6213
feat(core): Expose xxx_options API by @Xuanwo in #6215
RFC-6209: Glob Support by @asukaminato0721 in #6209
feat(bindings/java): Add WriteOptions support for new options API by @geruh in #6219
feat(services/azdls): Support parsing Azure Storage configs from connection strings by @DerGut in #6212
feat(bindings/java): Add ListOptions support for new options API by @geruh in #6246
feat(bindings/python): Enhance Reader and Writer by @chitralverma in #6086
feat(bindings/java): Add StatOptions support for new options API by @geruh in #6255
feat(website): Auto-generate llms.txt and llms-full.txt by @kingsword09 in #6247
oli: support dropbox by @Kinchkun in #6265
feat(bindings/python): Enhance Stat, Lister, Metadata & Entry by @chitralverma in #6232
feat(layers): add fastmetrics layer by @koushiro in #6269
feat(bindings/haskell): add more api by @asukaminato0721 in #6264
feat(core): Expose presign_xxx_options API by @geruh in #6273
feat: Add HttpClientLayer to replace Operator::update_http_client() by @Xuanwo in #6290
feat(bin/oli): support oli edit by @asukaminato0721 in #6229
feat(bindings/cpp): cpp async op && reader, lister by @asukaminato0721 in #6228
feat(services/moka): expose more moka configurations by @koushiro in #6285
feat(bindings/nodejs): Add StatOptions support for new options API by @kingsword09 in #6282
feat: implement --tree option for oli ls subcommand by @waynexia in #6311
feat(bindings/nodejs): Add ReadOptions and ReaderOptions support for new options API by @kingsword09 in #6312
feat(bindings/nodejs): Add ListOptions support for new options API by @kingsword09 in #6320
feat(bindings/go): add benchmark by @yuchanns in #6341
feat(services/azdls): Implement write returns metadata by @jonathanc-n in #6368
feat(bindings/cpp): remove Boost dependency by @JackDrogon in #6376
feat(bindings/nodejs): Add DeleteOptions support for new options API by @kingsword09 in #6349
feat(bindings/nodejs): Add WriteOptions support for new options API by @kingsword09 in #6322
feat(services/vercel_blob): add delete operator by @kingsword09 in #6396

Changed

refactor(core/types)!: fix with_user_metadata signature by @meteorgan in #5960
refactor(!): Remove services lack of maintainers and users by @Xuanwo in #6263
refactor(services/moka)!: replace sync::Cache with future::Cache by @koushiro in #6270
refactor(bindings/go): Restructure FFI system with type-safe wrapper by @yuchanns in #6268
refactor: Migrate redis from adapter::kv to Access instead by @Xuanwo in #6291
refactor: Migrate moka from adapter::typed_kv to Access instead by @Xuanwo in #6300
refactor: Migrate memory service to implment Access directly by @Xuanwo in #6301
refactor: Migrate services cacache to implement Access by @Xuanwo in #6303
refactor!: Remove stat_has_xxx and list_has_xxx by @Xuanwo in #6313
re...

Contributors

erickguan, kezhuw, and 23 other contributors

Assets 2

0 Join discussion

17 Jul 06:39

github-actions

v0.48.3

8ed2692

v0.48.3

refactor(website): remove unused dependencies and update node engine …

Assets 2

27 May 06:56

Xuanwo

v0.53.3

4e00288

v0.53.3

What's Changed

Added

feat(java): add ReadOptions for read methods by @liamzwbao in #6157
feat(core): support sharing one redb database between different Operators by @TD-Sky in #6173
feat(bin/oli): support cp to dir by @asukaminato0721 in #6140
feat(bin/oli): support tee by @asukaminato0721 in #6194
feat: expose Error::backtrace() by @xxchan in #6196
feat(bindings/c)!: Make features configurable via CMakeLists by @asukaminato0721 in #6143
feat: ADLS Client Credential Authentication by @c-thiel in #6205

Changed

refactor(bindings/java): deprecate append in favor of write with append=true by @kingsword09 in #6169
refactor(core): Put Backtrace in a box to reduce error size by @Xuanwo in #6193

Fixed

fix(bindings/go): update dependencies by @JupiterRider in #6175
fix(integrations/object_store): fix double percent encoding by @Colerar in #6166
fix(services/fs): Make fs services work on our MSRV by @Xuanwo in #6183
fix(services/s3): Remove not needed check for batch delete by @Xuanwo in #6206

Docs

fix: typo error in 0429_init_from_iter.md by @warjiang in #6203
doc: case-police Ocaml to OCaml by @yihong0618 in #6204

CI

ci: uses taiki-e/install-action to replace cargo insatll by @xxchan in #6168
ci(bindings/node): Add contents permissions for nodejs release by @Xuanwo in #6182
ci(bindings/java): Enable zig build for java by @Xuanwo in #6181
ci: Fix behavior tests for go bindings by @Xuanwo in #6199

Chore

Bump opendal to v0.53.3 (Round 1) by @Xuanwo in #6186
chore: remove useless labeler.yml by @koushiro in #6192

New Contributors

@liamzwbao made their first contribution in #6157
@kingsword09 made their first contribution in #6169
@JupiterRider made their first contribution in #6175
@TD-Sky made their first contribution in #6173
@Colerar made their first contribution in #6166
@warjiang made their first contribution in #6203
@c-thiel made their first contribution in #6205

Full Changelog: v0.53.2...v0.53.3

Contributors

Xuanwo, warjiang, and 10 other contributors

Assets 2

0 Join discussion

13 May 09:31

Xuanwo

v0.53.2

dfb1940

v0.53.2

What's Changed

Added

feat(core): impl Drop for BlockingWrapper by @asukaminato0721 in #6036
feat(bindings/python): add check in py by @asukaminato0721 in #5973
feat(core): Use divan for benchmark framework by @Xuanwo in #6051
feat(bindings/python): Add start_after support for list by @asukaminato0721 in #6054
feat(services/upyun): Add operation in http context by @jorgehermo9 in #6062
feat(services/alluxio): Add operation in http context by @jorgehermo9 in #6065
feat(services/cos): Add operation in http context by @jorgehermo9 in #6067
feat(services/github): Add operation in http context by @tks1197 in #6069
feat(services/s3): Support request payer option by @antoninferrand in #6070
feat(layers/prometheus_client): Add disable_label_root to allow skip root label in metrics by @flaneur2020 in #6071
feat(bindings/python): export MimeGuessLayer by @pk5ls20 in #6073
feat(services/obs): Add operation in http context by @tks1197 in #6079
feat(integrations/object_store): add AmazonS3Builder by @meteorgan in #5456
feat(integration/object_store): bump object store version by @XiangpengHao in #6091
feat(services/pcloud): Add operation http context by @jorgehermo9 in #6092
feat(services/vercel_artifacts): Add operation http context by @jorgehermo9 in #6093
feat(services/aliyun_drive): Add operation in http context by @jorgehermo9 in #6063
feat(services/ghac): Add operation http context by @jorgehermo9 in #6100
feat(services/b2): Add operation in http context by @jorgehermo9 in #6066
feat(services/webdav): Add operation in http context by @jorgehermo9 in #6107
feat(services/swift): Add operation in http context by @jorgehermo9 in #6106
feat(services/vercel_blob): Add operation in http context by @jorgehermo9 in #6101
feat(services/gdrive): Add operation in http context by @jorgehermo9 in #6068
feat(services/koofr): Add operation http context by @jorgehermo9 in #6096
feat(services/seafile): Add operation in http context by @jorgehermo9 in #6105
feat(services/yandex_disk): Add operation in http context by @jorgehermo9 in #6064
feat(services/webhdfs): Add operation in http context by @jorgehermo9 in #6112
feat(services/onedrive): Add operation in http context by @jorgehermo9 in #6111
feat(bindings/python): build and publish musllinux wheels (#6114) by @zhu0629 in #6116
feat(services/azblob): add request context by @erickguan in #6121
feat(services/azdls): add request context by @erickguan in #6122
feat(bindings/c): Add seek support for c-binding by @taoseng in #6119
feat(bindings/cpp): eliminate indirect pointers by manually managing memory by @deadlinefen in #6147
refactor(bindings/c)!: extract the writer closing logic from opendal_writer_free by @taoseng in #6128
feat(bindings/go): implement io.Seeker by @yuchanns in #6151

Changed

refactor(core): Deprecate not used options in OpList by @Xuanwo in #6050
refactor(services/gdrive): move raw requests to core by @jorgehermo9 in #6088
refactor(services/ghac): move raw http calls to core by @jorgehermo9 in #6095
refactor(services/yandex_disk): Move raw request to core by @jorgehermo9 in #6090
refactor(services/azdls): Refactor raw request send in writer and bac… by @jorgehermo9 in #6109
refactor(services/seafile): Refactor raw request send in writer and lister by @jorgehermo9 in #6104
refactor(services/azblob): Refactor raw request send in writer by @jorgehermo9 in #6102
refactor(services/aliyun_drive): Move raw request to core by @jorgehermo9 in #6089
refactor(services/webhdfs): Refactor raw request send in writer and backend by @jorgehermo9 in #6113
refactor(bindings/go): add ffiCall type for FFI function signature by @yuchanns in #6158

Fixed

fix(core): Only run size tests on 64bit platforms by @cryptomilk in #6078
fix(bindings/go): Reader returns io.EOF at the end of file by @yuchanns in #6150
fix(core/services/fs): Returning empty dir while list a file path by @Xuanwo in #6154

Docs

doc(services/onedrive): add a comment about onedrive's writer by @erickguan in #6120

CI

ci(bindings/go): include sqlite service into behavior tests by @yuchanns in #6039

Chore

chore: delete unused code in scripts by @yihong0618 in #6084
chore(integrations/unftp): Make compatible with latest libunftp by @hannesdejager in #6094
chore(deps): bump logforth from 0.23.1 to 0.24.0 in /bin/ofs by @dependabot in #6137
chore(deps): bump assert_cmd from 2.0.16 to 2.0.17 in /bin/oli by @dependabot in #6138
chore(deps): bump logforth from 0.23.1 to 0.24.0 in /bin/oay by @dependabot in #6139
chore(deps): update dav-server requirement from 0.7.0 to 0.8.0 in /integrations/dav-server by @dependabot in #6133
chore(deps): bump golangci/golangci-lint-action from 6 to 7 by @dependabot in #6134

New Contributors

@deadlinefen made their first contribution in #6041
@frex-is made their first contribution in #6057
@tks1197 made their first contribution in #6069
@antoninferrand made their first contribution in #6070
@pk5ls20 made their first contribution in #6073
@cryptomilk made their first contribution in #6078
@hannesdejager made their first contribution in #6094
@zhu0629 made their first contribution in #6116
@taoseng made their first contribution in #6119
@phreed made their first contribution in #6130

Full Changelog: v0.53.1...v0.53.2

Contributors

flaneur2020, phreed, and 18 other contributors

Assets 2

0 Join discussion

19 Apr 04:01

Xuanwo

v0.53.1

551b361

v0.53.1

What's Changed

Added

feat(services/gcs): implement write returns metadata by @meteorgan in #5933
feat(bindings/ruby): support layers by @erickguan in #5874
feat(gridfs): implement GridfsCore for GridFS service by @uruemu in #5966
feat(bindings/nodejs): add check in js by @asukaminato0721 in #5996
feat(core): Use Buffer as http_body::Body directly by @Xuanwo in #6026

Changed

refactor: Remove dead code ConcurrentFutures by @Xuanwo in #5939
refactor(layers/tracing): Ensure the entire async function been traced by @Xuanwo in #6000

Fixed

fix(core): Fix head-of-line blocking in concurrent tasks by @Xuanwo in #5941
fix(services/ipmfs): fix Ipmfs behavior tests by @miroim in #5969
chore(core): Fix clippy for services azfile by @asukaminato0721 in #5994

Docs

docs(bindings/nodejs): fix broken links by @miroim in #6017
doc(chore): update copyright date in NOTICE by @caicancai in #6023

CI

ci: add test for cpp example by @silver-ymz in #5952
ci(bindings/go): include go binding into behavior tests by @yuchanns in #6018

Chore

chore: rm in favor of dosubot by @asukaminato0721 in #5946
chore: Fix clippy for rust 1.86 by @Xuanwo in #5951
chore: sync cpp example by @silver-ymz in #5944
chore: update typos version by @yihong0618 in #5954
chore(core): fix typos for the QueryPairsWriter by @kemingy in #5978
chore(service/azdls): use QueryPairsWriter for url write by @kemingy in #5988
chore(services/obs): Use QueryPairsWriter for url write by @asukaminato0721 in #5985

New Contributors

@uruemu made their first contribution in #5949

Full Changelog: v0.53.0...v0.53.1

Contributors

erickguan, meteorgan, and 9 other contributors

Assets 2

0 Join discussion

05 Apr 03:29

Xuanwo

v0.53.0

12a076c

v0.53.0

Upgrade to Rust Core v0.53

Public API

Supabase service is now an S3-compatible servcice

Supabase Storage is now an S3-compatible service instead: https://github.com/supabase/storage.

We removed the supabase native service support in OpenDAL v0.53. Users who want to access Supabase Storage can use the S3 service instead.

All metrics related layers have been refactored

All metrics layers have been refactored:

PrometheusLayer
PrometheusClientLayer
MetricsLayer

They are now provides more metrics and more detailed information. All their public API have been redesigned.

For more details, please refer to opendal::layers::observe's module documentation.

`Operator::default_executor` has been replaced by `Operator::executor`

In opendal v0.53, we introduced a new concept of Context which is used to store the context of the current operator. Thanks to this design, we can now get and set the executor and http_client for given Operator instead.

All services http_client API has been deprecated and replaced by Operator::update_http_client API.

OpenDAL MSRV bumped to `1.80`

Since v0.53, OpenDAL will require Rust 1.80.0 or later to build.

Raw API

Operation enum merge

To reduce the complexity of the Operation, we have merged the duplicated Operation.

For example:

Operation::ReaderRead has been merged into Operation::Read
Operation::BlockingRead has been merged into Operation::Read

Upgrade to Nodejs Binding v0.48

Breaking change

Public API

Now, nodejs binding op.is_exist changed to op.exists to align with nodejs API style.

What's Changed

Added

feat(bindings/python) detailed error message by @asukaminato0721 in #5646
feat(bindings/python): Add user metadata support for write by @Xuanwo in #5654
feat(services/s3): add append support by @Frank-III in #5428
feat(core): Add presign delete support by @asukaminato0721 in #5647
feat(bindings/python): add python presign_delete by @asukaminato0721 in #5661
feat(core): Sharing context between layers and service by @Xuanwo in #5662
feat(!): Bump arrow version of parquet_opendal to 54.x by @erickguan in #5665
feat: Polish context related APIs by @Xuanwo in #5673
feat(services/onedrive): List dir shows metadata by @erickguan in #5632
feat(java): add WriteOptions for write methods by @geruh in #5664
feat(bindings/ruby): add operator info by @erickguan in #5584
feat(services/oss): Implement Write Returns Metadata for oss by @meteorgan in #5688
feat(core): Expose Reader::into_stream and Writer::into_sink by @Xuanwo in #5698
feat: adopt uv for python binding by @kemingy in #5711
feat(services/gcs): Implement multipart/related and use it with Gcs by @wlinna in #5691
feat(bindings/dart): Add dart binding by @asukaminato0721 in #5591
fix(binding/dart): pin version to avoid generator/runtime version mismatch by @asukaminato0721 in #5734
feat(binding/dart): add examples, tests by @asukaminato0721 in #5740
feat(services/onedrive): add signer to utilize the refresh token by @erickguan in #5733
feat(binding/dart): hide init && change to init by @asukaminato0721 in #5742
feat(core): Implement write returns metadata for b2 by @hoslo in #5750
feat(services/hdfs_native): implement write/read/list methods by @zhaohaidao in #5617
feat(website): Adding a WIP to the list of bingdings by @shaonianche in #5769
feat(services/onedrive): implement read_with_if_none_match by @erickguan in #5763
feat(binding/dart): add pubspec info by @asukaminato0721 in #5751
feat(services/s3): Add operation in http context by @Xuanwo in #5791
feat(bindings/python): Add repr for metadata by @yihong0618 in #5783
feat: Add origin private file system scaffold by @Eason0729 in #5758
feat(core): Implement http related metrics support for prom client by @Xuanwo in #5798
feat(core): Implement http related metrics support for otel by @leiysky in #5800
feat(services/lakefs): Add operation in http context by @liugddx in #5809
feat(github): Auto generate weekly summary by @Xuanwo in #5818
feat(binding/java): Add list with recursive support by @cuichenli in #5718
feat(services/huggingface): Add operation in http context by @liugddx in #5810
feat(core/layers): implement http related metrics support for prometheus by @koushiro in #5847
feat(core/layers): implement http related metrics support for metrics by @koushiro in #5848
feat(services/onedrive): implement additional OneDrive features by @erickguan in #5784
feat(bindings/c): add opendal_operator_check by @asukaminato0721 in #5851
feat(services/aliyun_drive): Add operation in http context by @sunheyi6 in #5880

Changed

refactor(!): Supabase is now an S3-compatible servcices by @Xuanwo in #5663
refactor: Migrate s3 services to context based http client by @Xuanwo in #5676
refactor: Migrate oss services to context based http client by @Ziy1-Tan in #5681
refactor: Migrate obs services to context based http client by @Ziy1-Tan in #5682
refactor: Migrate cos services to context based http client by @Ziy1-Tan in #5683
refactor(bindings/node)!: Change is_exist to exists to align with nodejs API style by @yihong0618 in #5731
refactor(services/redis): Implement ConnectionLike for RedisConnection by @Xuanwo in #5748
refactor!: Remove opendal-compat which is not maintained by @Xuanwo in #5754
refactor: Migrate github services to context based http client by @miroim in #5764
refactor(gcs): Migrate to context based http client by @leiysky in #5778
refactor(core/raw)!: Merge blocking and async operations by @Xuanwo in #5789
refactor(core/raw)!: Use AccessorInfo instead of seperate fields by @Xuanwo in #5796
refactor: Migrate aliyun_drive services to context based http client by @miroim in #5815
refactor: Migrate azfile services to context based http client by @miroim in #5816
refactor: Migrate dropbox services to context based http client by @miroim in #5827
refactor: Migrate upyun services to context based http client by @miroim in #5829
refactor(core): Migrate BlockWriter to use executor from context by @Xuanwo in #5834
refactor(core): Migrate MultipartWrite Executor to context based by @Xuanwo in #5835
refactor(core): Migrate PositionWrite Executor to context based by @Xuanwo in #5836
refactor(core)!: Migrate to context based executor by @Xuanwo in #5838
refactor: tidy binding java code by @tisonkun in #5840
refactor: Migrate yandex_disk services to context based http client by @miroim in #5841
refactor: Migrate alluxio services to context based http client by @miroim in #5842
refactor: Migrate b2 service to context based http client by @miroim in #5843
refactor: migrate azblob services to context based http client by @Ziy1-Tan in #5845
refactor: Migrate gdrive service to context based http client by @miroim in #5861
refactor: Migrate pcloud service to context based http client by @miroim in #5866
refactor!: Bump OpenDAL MSRV to 1.80 by @Xuanwo in #5868
refactor: Introduce VercelArtifactsCore for improved service structure by @miroim in #5873
refactor: Migrate vercel_artifacts service to context based http...

Contributors

erickguan, meteorgan, and 25 other contributors

Assets 2

0 Join discussion

24 Feb 06:57

Xuanwo

v0.52.0

45e754b

v0.52.0

Upgrade to v0.52

Public API

RFC-5556: Write Returns Metadata

Since v0.52, all write APIs in OpenDAL have been updated to return Metadata instead of (). This metadata includes useful information provided by the service, such as content-length, etag, version, and last-modified.

This feature is not fully ready yet, and many available metadata fields are still not returned. Please visit Tracking Issues of RFC-5556: Write Returns Metadata for progress and contributions.

Affected API:

opendal::Operator::write
opendal::Operator::write_with
opendal::Operator::writer::close
opendal::raw::oio::Write::close

Github Actions Cache (ghac) service v2

As requested by GitHub, we have upgraded our GHAC service to ensure compatibility with the latest GitHub Actions cache API.

By upgrading to OpenDAL v0.52, your services will continue functioning after the deprecation of the legacy service (2025/03/01). GHES does not yet support GHAC v2, but OpenDAL has handled this properly to prevent any disruptions.

ghac service doesn't support delete anymore, please use github's API to delete cache instead.

This upgrade is mandatory and enabled by default using an environment variable in the GitHub CI environment. No changes are required at the code level.

Breaking Changes in Dependencies

OtelTraceLayer and OtelMetricsLayer's dependence opentelemetry bumped to 0.28
PrometheusClientLayer's dependence prometheus-client bumped to 0.23.1

v0.52.0

Added

feat(services/s3): Added crc64nvme for s3 by @geetanshjuneja in #5580
feat(services-fs): Support write-if-not-exists in fs backend by @SergeiPatiakin in #5605
feat(services/gcs): Impl content-encoding support for GCS stat, write and presign by @wlinna in #5610
feat(bindings/ruby): add lister by @erickguan in #5600
feat(services/swift): Added user metadata support for swift service by @zhaohaidao in #5601
feat: Implement github actions cache service v2 support by @Xuanwo in #5633
feat(core)!: implement write returns metadata by @meteorgan in #5562
feat(bindings/python): let path can be PathLike by @asukaminato0721 in #5636
feat(bindings/python): add exists by @asukaminato0721 in #5637

Changed

refactor: Remove dead services libsql by @Xuanwo in #5616

Fixed

fix(services/gcs): Fix content encoding can't be used alone by @Xuanwo in #5614
fix: ghac doesn't support delete anymore by @Xuanwo in #5628
fix(services/gdrive): skip the trailing slash when creating and querying the directory by @meteorgan in #5631

Docs

docs(bindings/ruby): add documentation for Ruby binding by @erickguan in #5629
docs: Add upgrade docs for upcoming 0.52 by @Xuanwo in #5634

CI

ci: Fix bad corepack cannot find matching keyid by @Xuanwo in #5603
ci(website): avoid including rc when calculate the latest version by @tisonkun in #5608
build: upgrade opentelemetry dependencies to 0.28.0 by @tisonkun in #5625
ci: Try fix nodejs CI by @Xuanwo in #5643
ci: Skip corepack checks until next nodejs version by @Xuanwo in #5644

Chore

chore(deps): bump uuid from 1.11.0 to 1.12.1 in /bin/oli by @dependabot in #5589
chore(deps): bump uuid from 1.11.0 to 1.12.1 in /core by @dependabot in #5588
chore(deps): bump log from 0.4.22 to 0.4.25 in /bin/oay by @dependabot in #5590
chore(deps): bump tempfile from 3.15.0 to 3.16.0 in /bin/ofs by @dependabot in #5586
chore(deps): update libtest-mimic requirement from 0.7.3 to 0.8.1 in /integrations/object_store by @dependabot in #5587
chore(layers/prometheus-client): upgrade prometheus-client dependency to v0.23.1 by @koushiro in #5576
chore(ci): remove benchmark report by @dqhl76 in #5626

New Contributors

@SergeiPatiakin made their first contribution in #5605
@wlinna made their first contribution in #5610
@zhaohaidao made their first contribution in #5601

Full Changelog: v0.51.2...v0.52.0

Contributors

erickguan, meteorgan, and 10 other contributors

Assets 2

0 Join discussion

07 Feb 01:26

tisonkun

v0.51.2

ca56238

v0.51.2

What's Changed

Added

feat(core): implement if_modified_since and if_unmodified_since for stat_with by @meteorgan in #5528
feat(layer/otelmetrics): add OtelMetricsLayer by @andylokandy in #5524
feat(integrations/object_store): implement put_opts and get_opts by @meteorgan in #5513
feat: Conditional reader for azblob, gcs, oss by @geetanshjuneja in #5531
feat(core): Add correctness check for read with if_xxx headers by @Xuanwo in #5538
feat(services/cos): Added user metadata support for cos service by @geetanshjuneja in #5510
feat(core): Implement list with deleted and versions for oss by @hoslo in #5527
feat(layer/otelmetrics): take meter when register by @andylokandy in #5547
feat(gcs): Convert TOO_MANY_REQUESTS to retryable Ratelimited by @Xuanwo in #5551
feat(services/webdfs): Add user.name support for webhdfs by @Xuanwo in #5567
feat: disable backtrace for NotFound error by @xxchan in #5577

Changed

refactor: refactor some unnecessary clone and use next_back to make clippy happy by @yihong0618 in #5554
refactor: refactor all body.copy_to_bytes(body.remaining()) by @yihong0618 in #5561

Fixed

fix(integrations/object_store) object_store_opendal now compiles on wasm32-unknown-unknown by @XiangpengHao in #5530
fix(serivces/gcs): Gcs doesn't support read with if_(un)modified_since by @Xuanwo in #5537
fix(logging): remove additional space by @xxchan in #5568

Docs

docs: Fix opendal rust core's README not align with new vision by @Xuanwo in #5541
docs(integration/object_store): add example for datafusion by @meteorgan in #5543
docs: Add docs on how to pronounce opendal by @Xuanwo in #5552
docs(bindings/java): better javadoc by @tisonkun in #5572

CI

ci(integration/object_store): add integration tests for object_store_opendal by @meteorgan in #5536
ci: Pin the nightly version to rust 1.84 for fuzz by @Xuanwo in #5546
ci: skip running behavior tests when adding or modifying documentation by @meteorgan in #5558
build: fix Cargo.lock and pass --locked in CI by @xxchan in #5565
build: implement release process in odev by @tisonkun in #5592
ci: upgrade upload and download artifact actions by @tisonkun in #5598

Chore

chore: Update CODEOWNERS by @Xuanwo in #5542
chore(layer/otelmetrics): take meter by reference by @andylokandy in #5553
chore(core): Avoid using mongodb 3.2.0 by @Xuanwo in #5560
chore: add oli/oay/ofs to rust-analyzer.linkedProjects by @xxchan in #5564
chore: try use logforth by @tisonkun in #5573
chore: bump version 0.51.2 by @tisonkun in #5595
chore: add changelog for 0.51.2 by @tisonkun in #5597

New Contributors

@XiangpengHao made their first contribution in #5530

Full Changelog: v0.51.1...v0.51.2

Contributors

meteorgan, Xuanwo, and 7 other contributors

Assets 2

Releases: apache/opendal

v0.55.0

Rust Core Upgrade to v0.55

Public API

Timestamp types now come from jiff

Scheme handling is string-based

List APIs only support versions

S3Builder::security_token removed

KV-style services no longer pretend to support list

Raw API

Deprecated KV adapters removed

Bindings Python Upgrade to v0.47

Breaking change: Module exports are explicit

Breaking change: Capability accessors renamed

Breaking change: Service identifiers now have typed enums

What's Changed

Added

Changed

Fixed

Contributors

Uh oh!

v0.54.1

What's Changed

Added

Changed

Fixed

Docs

CI

Chore

New Contributors

Contributors

Uh oh!

v0.54.0

Upgrade Notes

Rust Core Public API

RFC-6189: Remove Native Blocking Support

RFC-6213: Options Based API

Remove stat_has_xxx and list_has_xxx APIs

Fix with_user_metadata signature

Services removed due to lack of maintainer

HttpClientLayer replaces update_http_client

Expose presign_xxx_options API

Rust Core Raw API

Remove native blocking support

Bindings Java Breaking changes

Removed services

Batch operations removed

Capability changes

Options-based API

Bindings Python

Breaking change: Native blocking API removed

Breaking change: Removed services

Breaking change: Chainsafe service removed

What's Changed

Added

Changed

Contributors

Uh oh!

v0.48.3

Uh oh!

v0.53.3

What's Changed

Added

Changed

Fixed

Docs

CI

Chore

New Contributors

Contributors

Uh oh!

v0.53.2

What's Changed

Added

Changed

Fixed

Docs

CI

Chore

New Contributors

Timestamp types now come from `jiff`

List APIs only support `versions`

`S3Builder::security_token` removed

KV-style services no longer pretend to support `list`

Remove `stat_has_xxx` and `list_has_xxx` APIs

Fix `with_user_metadata` signature

HttpClientLayer replaces `update_http_client`

Expose `presign_xxx_options` API

`Operator::default_executor` has been replaced by `Operator::executor`

OpenDAL MSRV bumped to `1.80`