v2.7.2

Arrow v2.7.2

Diff since v2.7.1

Merged pull requests:

• Compatibility of schemas with nested types (#504) (@poncito)

Closed issues:

• Arrow-over-HTTP client and server examples in Julia (#502)
• snappy_jll v1.2.0 lead to Arrow_jll failed to build (#505)

v2.7.1

Arrow v2.7.1

Diff since v2.7.0

Merged pull requests:

• Bugfix type instability in length(::Arrow.ToList) (#497) (@JoaoAparicio)

v2.7.0

Arrow v2.7.0

Diff since v2.6.2

Merged pull requests:

• cleanup testing (#462) (@baumgold)
• Formatting (#464) (@baumgold)
• Ensure that ArrowTypes.default is defined for Vararg tuples (#466) (@quinnj)
• Fix typo in passing largelists keyword arg (#467) (@quinnj)
• Try to fix docs by publishing to apache infra (#468) (@quinnj)
• Use wkspawn from ConcurrentUtilities instead of Threads.spawn (#469) (@quinnj)
• Fix documentation link (#478) (@guilhermebodin)
• Fix documentation example for Arrow.Writer (#479) (@guilhermebodin)
• Fix example in Arrow.Writer (#480) (@guilhermebodin)
• Docs: add a note about using copy() to get a DataFrame where the columns are regular Vectors (#487) (@DilumAluthge)
• bump (#488) (@JoaoAparicio)
• Use https://arrow.apache.org/julia/ as the official document URL (#490) (@kou)
• Fix docs errors (#491) (@ericphanson)
• enable field-order-agnostic overloads of fromarrow for struct types (#493) (@jrevels)

Closed issues:

• Loss of parametric type information for custom types (#134)
• Look into potential over-copying when copying arrow vector (#190)
• Arrow.write with partition/partitioner results batch unique value error (#192)
• Roundtrippability of special strings (#196)
• Support InlineStrings (#304)
• Invalid argument error (#336)
• ERROR: MethodError: no method matching zero(::Type{Union{Nothing, String}}) when trying to serialize Union{Nothing,String} fields (#368)
• github releases not in sync (#370)
• Typo in kwargs of arrowvector for ListKind (#392)
• Introduce automatic code formatting with JuliaFormatter.toml (#398)
• Issue with Union{Missing, VersionNumber} (#461)
• GitHub Pages build error (#463)
• Use https://arrow.apache.org/julia/ as the official Website URL (#470)
• Arrow.write performance on large DataFrame (#473)
• getindex broken with SVector{3, UInt} in the presence of missing data (#486)
• Removing .arrow files without closing Julia seems impossible in Windows (#492)

v2.6.2

Arrow v2.6.2

Diff since v2.6.1

Closed issues:

• Does this pkg provide the plasma In-Memory Object Store? (#37)
• Allow for file appends (#105)
• Invalid arrow file produced when writing partitioned with a dictionary-encoded column? (#129)
• New package release process (#279)
• Need to improve code review/release process and reduce developer productivity friction (#342)
• [Feature Request] Support Tensors ? (#433)
• Error with v2.6.0 (#458)

Merged pull requests:

• implement default for SubArray (#456) (@baumgold)
• Fix case of recursively nested lists (#459) (@quinnj)
• Bump Arrow & ArrowTypes versions (#460) (@quinnj)

v2.6.1

Arrow v2.6.1

Diff since v2.6.0

Closed issues:

• Release document misses how to register ArrowTypes to the Julia General Registry (#450)
• Arrow.jl 2.6 breaks Legolas.jl's tests (#452)
• Incorrect syntax in ArrowTypes code (#453)

Merged pull requests:

• release: Add how to register ArrowTypes to the Julia General Registry (#451) (@kou)
• #453 fix typeo introduced in #389 (#454) (@baumgold)
• bump version for new release (#455) (@baumgold)

v2.6.0

Arrow v2.6.0

Diff since v2.5.2

Closed issues:

• Support Tables.partitions when reading "arrow file" format in addition to "stream" format (#293)
• Make recursive iteration to get dictionaries more defensive for interop (#375)
• Error/Segfault when writing many partitions (#396)
• Vector{UInt8} mis-represented when writing to disk (#411)
• CI doesn't test with multiple threads (#426)
• Malformed file by Arrow.write on a IOStream created with open(filename, "w") (#432)
• Unhandled sentinel value for len in compression causes invalid Array dimensions (#435)

Merged pull requests:

• Get dictionaries of children only when field.children not nothing (#382) (@okartal)
• fix Base.eltype methods and functions that take Type parameters (#404) (@baumgold)
• enable incremental reads of arrow-formatted files (#408) (@baumgold)
• Base.isdone for Stream (#428) (@baumgold)
• Run with 1 and 2 threads during tests (#431) (@quinnj)
• Add handling of len = -1 in uncompress (#436) (@DrChainsaw)
• Don't treat Vector{UInt8} as Arrow Binary type (#439) (@quinnj)
• Bump BitIntegers compat (#441) (@quinnj)
• Handle len of -1 in "compressed" buffers from other languages (#442) (@quinnj)
• Add Tables.partitions definition for Arrow.Table (#443) (@quinnj)
• Remove scopedenum for EnumX (#444) (@quinnj)
• Refactor compressors/decompressors for laziness + safety (#445) (@quinnj)
• Return SubArrays when possible for arrow list types (#446) (@quinnj)
• bump version of Arrow and ArrowTypes to prepare for new release (#447) (@baumgold)

v2.5.2

Arrow v2.5.2

Diff since v2.5.1

Merged pull requests:

• Bump Arrow + ArrowTypes versions (#407) (@baumgold)

v2.5.1

Arrow v2.5.1

Diff since v2.5.0

Closed issues:

• Performance regression caused by Arrow type piracy (#266)
• Precompilation broken on Julia 1.9-rc1 (#391)
• Does Arrow.write have an upper limit for the number of columns? (#414)
• [Discussion] Need for early-returning friendly iteration interface (#417)
• Test failure on 1.8.5 Rosetta (#425)
• Test Failure (#429)

Merged pull requests:

• Fix deprecation of getproperty(::Pairs, s) (#409) (@baumgold)
• Use getproperty instead of type parameter to get names (#415) (@simsurace)
• Add @testsets for misc tests (#421) (@JoaoAparicio)
• Pre-allocate buffer (#422) (@JoaoAparicio)
• Bump patch version (#423) (@simsurace)
• add dev CI for monorepo (#424) (@Moelf)
• #429 fix alignment for compressed data (#430) (@baumgold)

v2.5.0

Arrow v2.5.0

Diff since v2.4.3

Closed issues:

• How to represent customize type field in schema (#383)
• Making Custom types documentation example work (#386)
• Tag release (#387)

Merged pull requests:

• Revert "Tag new version dev/release/release.sh (#381)" (#388) (@kou)
• fix arrowname and arrowmetadata for nullable types (#389) (@baumgold)
• add IPv4/IPv6 support to ArrowTypes (#390) (@baumgold)
• define eltype for Stream (#394) (@baumgold)
• Implement Tables.columnnames and Tables.schema for Arrow.Stream (#395) (@baumgold)
• Bump Arrow version to 2.5.0 (#406) (@baumgold)

v2.4.3

Arrow v2.4.3

Diff since v2.2.1

Closed issues:

• Re-use PyArrow memory via PyCall (#92)
• Can't roundtrip NamedTuples of Nanoseconds twice (#214)
• Serializing Dict{String,Real} result in garbage values (#232)
• VersionNumber issues when you also have missing (#243)
• Order of record batches from "arrow file" format files (i.e. Arrow.Table) not preserved (#295)
• dev/release/release.sh misses a feature that removes old releases and RCs (#307)
• Release process misses "add a new release to ASF's report database" (#311)
• RC verification script name is wrong (#313)
• Verify release CI jobs are failed for a RC1 commit (#314)
• Versions in footer and message do not agree, this causes issues reading Arrow files with other libraries (such as arrow-rs). (#320)
• filtering DataFrame loaded from feather file triggers deleteat! error (#324)
• DST ambiguities in ZonedDateTime not supported (#327)
• error earlier when number of entries don't match across all fields (#344)
• Tests fail on Apple silicon on Julia 1.8 (#345)
• Install Registrator.jl github app (#348)
• Allow appending record batches to an existing Arrow file (#352)
• Arrow.append to non-existent file (#354)
• PooledArray are incorrectly saved (#364)
• inappropriately applied Arrow.NullVector optimization to Union{ZonedDateTime,Missing} column (#367)
• Release [email protected] (#376)
• RC verification script doesn't test ArrowTypes (#378)
• dev/release/release.sh doesn't tag (#380)

Merged pull requests:

• refactor Arrow.write to support incremental writes (#277) (@baumgold)
• disallow non-concrete map-like types to prevent incorrect serialization (#305) (@jrevels)
• add missing arrowtype(b, ::Type{<:Period}) method to enable roundtripping of Period types (#306) (@jrevels)
• Bump version to 2.3.0 (#312) (@quinnj)
• Remove old releases and RCs on a new release (#315) (@kou)
• Use a large RC number for CI (#316) (@kou)
• Fix wrong RC verification path (#317) (@kou)
• Add a missing release process to update ASF's report database (#318) (@kou)
• fix version mismatch by changing footer to V5 (#321) (@pcjentsch)
• Fix repo name in the Readme (#322) (@CarloLucibello)
• allow ntasks to be 0 and determine whetheher to use threads based on nthreads rather than ntasks (#325) (@baumgold)
• Store ZDT with a UTC, not local, timestamp (#329) (@tpgillam)
• support reading empty files that are in the process of being written (#338) (@baumgold)
• Use OrderedSynchronizer instead of OrderedChannel (#339) (@quinnj)
• Add ArgumentError for deleteat on ArrowVectors; fixes #324 (#341) (@quinnj)
• Remove requirement to have 1 approval on PRs (#343) (@quinnj)
• Bump version to 2.4.0 (#346) (@baumgold)
• Tar is also needed to verify release candidates (#347) (@bkmgit)
• fix Julia 1.6 (#349) (@SimonDanisch)
• replace custom debug handling with LoggingExtras (#355) (@baumgold)
• fix BatchIterator iterate method to handle partial messages (#356) (@baumgold)
• Ensure Julia types have alignment respected (#357) (@quinnj)
• allow append to act on non-existent/blank file/io (#358) (@tanmaykm)
• don't run tests that fail on Windows (#361) (@visr)
• Bump compat for LoggingExtras for 1.0 (#362) (@quinnj)
• Ensure elements are converted when indexed from ArrowTypes.ToArrow (#365) (@quinnj)
• MINOR: Update issue notifications to issues mailing list to match Arrow repo (#366) (@raulcd)
• Create CompatHelper.yml (#369) (@CarloLucibello)
• Fix issue with missing and non-concrete Arrow types (#371) (@omus)
• Define defaults for Missing/Nothing (#372) (@omus)
• bugfix & test for columns with VersionNumber & missing (#374) (@ericphanson)
• Bump Arrow version to 2.4.2 (#377) (@quinnj)
• Test ArrowTypes package in RC verification script (#379) (@kou)
• Tag new version dev/release/release.sh (#381) (@kou)
• Fix ToArrow when using an abstract eltype with all missing elements (#385) (@omus)