Recent Releases of https://github.com/rails/marcel
https://github.com/rails/marcel - v2.1.0
Marcel 2.1.0 refreshes the MIME database to the Apache Tika 4.0.0 release (#177). No API, Ruby requirement, canonical type, alias, or extension changes.
New detection
- Android binary XML (
application/vnd.android.axml) — compiledAndroidManifest.xmlandres/*.xmlfiles from APKs, previouslyapplication/octet-stream.
Fixes
application/timestamped-data(RFC 5544) content detection now recognizes definite-length DER encodings, not just the indefinite-length form.- Sibling CMS content types in the
1.2.840.113549.1.9.16.1.*arc (compressedData, authData, …) are no longer misidentified asapplication/timestamped-data.
- Ruby
Published by jeremy 3 months ago
https://github.com/rails/marcel - v2.0.0
Marcel 2.0.0 modernizes the toolchain, hardens the release pipeline, and teaches the content sniffer several new families of formats. It is a major release because it drops old Rubies and changes the canonical type returned for some inputs — details below so you can scan for the ones that affect you.
Breaking changes
- Ruby 3.3 or newer is required (was 2.3). (#170)
- Canonical type changes — the same bytes or filename may now return a different (more standard) type:
- Aliased types now resolve to their canonical form on output (#171):
audio/x-aac→audio/aac,audio/x-flac→audio/flac,audio/vnd.wave→audio/x-wav,text/x-yaml→application/yaml,application/x-debian-package→application/vnd.debian.binary-package,application/x-xliff+xml→application/xliff+xml. Lookups by any alias still work. image/bmp;format=compressedis now plainimage/bmp(#148)..cr2files areimage/x-canon-cr2instead ofimage/x-raw-canon(#172).- OpenXPS (
.oxps) is split from XPS (#171). - The XML/HTML detection work below can relabel documents that previously fell back to
application/xml,text/html, orapplication/octet-stream.
- Aliased types now resolve to their canonical form on output (#171):
Security & hardening
A review of the detection pipeline landed as a series of PRs (#157–#164):
- MIME data is validated before table generation, and the generated tables are checked in CI (#157,
rake tables:check). - HTML and XHTML magic matches only bounded opening-tag patterns instead of unanchored scans (#158).
- IO edge cases — partial reads, unseekable and unrewindable IOs, non-string input — are handled explicitly while sniffing (#159).
- Declared MIME types are validated rather than trusted (#160).
- Marcel's detection boundary — what sniffing does and does not promise — is documented in the README (#161).
- CI runs with locked dependencies (#163), and the release workflow builds and publishes the same verified gem artifact with sha256 recorded and sigstore attestation (#164), with the gem's contents verified by script (#162, #173).
New detection
- OOXML documents (
.docx/.xlsx/.pptxand their macro-enabled variants) are identified from the ZIP central directory, regardless of which application produced them (#169). - XML vocabularies are identified by their root element, the way Apache Tika does: RSS, Atom, KML, Apple property lists, XSLT, ONIX, iWork flat XML, Office 2003 XML, and friends (#173 and #175, fixing #48).
- Canon CR2 raw images (#172).
- RAR archives (#155, @tristan-f-r) and Google Sheets shortcuts (#156, @tristan-f-r).
- AVIF image sequences (
avisbrand) (#151, @mm12) and audio-only.m4afiles (#147). - HTML detection also checks the end of the document for closing tags (#146).
Performance
- The common-type fast path is restored with subtype-aware ordering, so common formats like JPEG are identified in ~25 IO reads instead of hundreds, without preempting subtype matchers (#172, #174).
- Sniffing uses
IO#seekwhere possible instead of reading and discarding, including for regexp magic (#144, #145).
Upgrading
Most applications need no changes beyond Ruby ≥ 3.3. If you compare Marcel's output against stored MIME types, re-check the canonical type changes above — lookups by the old aliases still resolve, but Marcel now returns the canonical name.
- Ruby
Published by jeremy 3 months ago
https://github.com/rails/marcel - v1.2.1
- Revert BMP images type to just
image/bmpinstead ofimage/bmp;format=compressed.
The later is more precise, but cause backward compatibility issues with Active Storage.
Full Changelog: https://github.com/rails/marcel/compare/v1.2.0...v1.2.1
- Ruby
Published by byroot 6 months ago
https://github.com/rails/marcel - v1.2.0
What's Changed
- Stop mutating source IO state during magic-byte detection by @andreaslillebo in https://github.com/rails/marcel/pull/143
- improve SVG detection by @alexanderadam in https://github.com/rails/marcel/pull/129
- improve HTML detection by @alexanderadam in https://github.com/rails/marcel/pull/130
- add fixture for BOM CSV by @alexanderadam in https://github.com/rails/marcel/pull/134
- add unicode string support by @alexanderadam in https://github.com/rails/marcel/pull/138
- Add support for Sony RAW image format magic detection by @bogdan in https://github.com/rails/marcel/pull/142
- Add support for .gpx files by @trekdemo in https://github.com/rails/marcel/pull/113
- add
tika.xmlregex support by @alexanderadam in https://github.com/rails/marcel/pull/132 - add pkcs8 detection by @alexanderadam in https://github.com/rails/marcel/pull/133
- Add hprof fixture and fix trailing space bug by @alexanderadam in https://github.com/rails/marcel/pull/136
New Contributors
- @andreaslillebo made their first contribution in https://github.com/rails/marcel/pull/143
- @alexanderadam made their first contribution in https://github.com/rails/marcel/pull/129
- @bogdan made their first contribution in https://github.com/rails/marcel/pull/142
- @trekdemo made their first contribution in https://github.com/rails/marcel/pull/113
Full Changelog: https://github.com/rails/marcel/compare/v1.1.1...v1.2.0
- Ruby
Published by byroot 6 months ago
https://github.com/rails/marcel - v1.1.1
What's Changed
- Fix Ruby 3.4 frozen string literal warnings with StringIO by @andynu in https://github.com/rails/marcel/pull/140
New Contributors
- @andynu made their first contribution in https://github.com/rails/marcel/pull/140
Full Changelog: https://github.com/rails/marcel/compare/v1.1.0...v1.1.1
- Ruby
Published by byroot 6 months ago
https://github.com/rails/marcel - v1.1.0
What's Changed
- Identify Sony and Canon raw images as subtypes of image/tiff by @afcapel in https://github.com/rails/marcel/pull/89
- Fix frozen string literal warning in magic detection by @FrancescoK in https://github.com/rails/marcel/pull/123
- Update tika definitions to latest version by @MarcelEeken in https://github.com/rails/marcel/pull/114
- Fix detection of AV1 in WebM as video/webm by @alexandergitter in https://github.com/rails/marcel/pull/104
New Contributors
- @afcapel made their first contribution in https://github.com/rails/marcel/pull/89
- @FrancescoK made their first contribution in https://github.com/rails/marcel/pull/123
- @MarcelEeken made their first contribution in https://github.com/rails/marcel/pull/114
- @Mth0158 made their first contribution in https://github.com/rails/marcel/pull/108
- @mark-young-atg made their first contribution in https://github.com/rails/marcel/pull/105
- @alexandergitter made their first contribution in https://github.com/rails/marcel/pull/104
- @rafaelfranca made their first contribution in https://github.com/rails/marcel/pull/126
Full Changelog: https://github.com/rails/marcel/compare/v1.0.4...v1.1.0
- Ruby
Published by rafaelfranca about 1 year ago
https://github.com/rails/marcel - v1.0.4
What's Changed
- Regression fix: binary declared type should fall back to filename extension type by @jeremy in https://github.com/rails/marcel/pull/99
Full Changelog: https://github.com/rails/marcel/compare/v1.0.3...v1.0.4
- Ruby
Published by jeremy over 2 years ago
https://github.com/rails/marcel - v1.0.3
What's Changed
- Prefer audio/ogg instead of audio/vorbis by @gmcgibbon in https://github.com/rails/marcel/pull/65
- Suppress warning by @wonda-tea-coffee in https://github.com/rails/marcel/pull/69
- Add explanation of MimeType.for's handling of argument types by @elebow in https://github.com/rails/marcel/pull/68
- tables.rb: Generate UTF-8 strings when possible. by @casperisfine in https://github.com/rails/marcel/pull/70
- Remove comment strings from Tables::TYPE by @casperisfine in https://github.com/rails/marcel/pull/71
- Store MIME parents in a distinct Hash by @casperisfine in https://github.com/rails/marcel/pull/72
- Fix magic detection for HTML with
<svgby @ursm in https://github.com/rails/marcel/pull/74 - Update gem name in Gemfile by @elebow in https://github.com/rails/marcel/pull/88
- Move to GitHub Actions by @hahmed in https://github.com/rails/marcel/pull/82
- Add note in README how to extend detection of custom file types by @vipulnsward in https://github.com/rails/marcel/pull/93
- Fix Illustrator detection as application/pdf instead of application/illustrator by @jeremy in https://github.com/rails/marcel/pull/94
New Contributors
- @wonda-tea-coffee made their first contribution in https://github.com/rails/marcel/pull/69
- @elebow made their first contribution in https://github.com/rails/marcel/pull/68
- @casperisfine made their first contribution in https://github.com/rails/marcel/pull/70
- @ursm made their first contribution in https://github.com/rails/marcel/pull/74
- @hahmed made their first contribution in https://github.com/rails/marcel/pull/82
- @vipulnsward made their first contribution in https://github.com/rails/marcel/pull/93
- @jeremy made their first contribution in https://github.com/rails/marcel/pull/94
Full Changelog: https://github.com/rails/marcel/compare/v1.0.2...v1.0.3
- Ruby
Published by jeremy over 2 years ago
https://github.com/rails/marcel - v1.0.2
- Include Apache license in gem release. (a525d5b38f287ca0511c8eb26e657a1d46686e5f)
- Prefer
audio/x-wavfor WAV audio files. (#45) - Prefer
application/x-x509-ca-certfor Privacy-Enhanced Mail certificates. (#46) - Prefer
audio/flacfor FLAC audio files. (#47) - Prefer
audio/aacfor Advanced Audio Coding audio files. (#49) - Prefer
application/vnd.ms-accessfor Microsodt Access DB files. (#50) - Support
text/x-scssandtext/x-sassstylesheets. (#52) - Support encrypted Microsoft Access DB files. (#53)
- Prefer
application/x-ole-storagefor Microsoft Office files. (#54) - Prefer
text/markdownfor Markdown files. (#55) - Prefer
audio/mpcfor Musepack audio files. (#56) - Support
audio/webmaudio files. (#58) - Support
image/avifimages files. (#63)
- Ruby
Published by rafaelfranca about 5 years ago
https://github.com/rails/marcel - v1.0.1
- Fixes identifying OpenDocument files by magic. 1.0.0 imprecisely identified them as
application/zip. (#38) - Fixes identifying
.docx,.pptx, and.xlsxfiles exported from Google Sheets by magic. (#36) - Identifies vCard files as
text/vcardrather thantext/x-vcard. (27fac74bd69663495410339bfd40ea8581ba669b) - Identifies
.otf,.woff, and.woff2files așfont/otf,font/woff, andfont/woff2, respectively. (#37)
- Ruby
Published by georgeclaghorn over 5 years ago
https://github.com/rails/marcel - v1.0.0
The mimemagic dependency—which relies on GPL-licensed mime type data from freedesktop.org’s shared-mime-info project—is removed. Marcel now directly uses mime type data adapted from the Apache Tika project, distributed under the Apache License.
- Ruby
Published by georgeclaghorn over 5 years ago
https://github.com/rails/marcel - v0.3.3
- Shush Ruby interpreter warnings (#6)
- Exclude test files from published gem to shrink it from 7.5 MB to 7.5 KB (#11)
- Ruby
Published by georgeclaghorn about 8 years ago
https://github.com/rails/marcel - v0.3.2
Emit the vnd/ms-powerpoint content type for *.ppt files. (#3)
- Ruby
Published by georgeclaghorn over 8 years ago
https://github.com/rails/marcel - v0.3.1
Support Ruby 2.2 for Rails compatibility.
- Ruby
Published by georgeclaghorn almost 9 years ago