A summary of data about the Ruby ecosystem.

Recent Releases of https://github.com/rails/marcel

https://github.com/rails/marcel - v2.1.0

Marcel 2.1.0 refreshes the MIME database to the Apache Tika 4.0.0 release (#177). No API, Ruby requirement, canonical type, alias, or extension changes.

New detection

  • Android binary XML (application/vnd.android.axml) — compiled AndroidManifest.xml and res/*.xml files from APKs, previously application/octet-stream.

Fixes

  • application/timestamped-data (RFC 5544) content detection now recognizes definite-length DER encodings, not just the indefinite-length form.
  • Sibling CMS content types in the 1.2.840.113549.1.9.16.1.* arc (compressedData, authData, …) are no longer misidentified as application/timestamped-data.

- Ruby
Published by jeremy 3 months ago

https://github.com/rails/marcel - v2.0.0

Marcel 2.0.0 modernizes the toolchain, hardens the release pipeline, and teaches the content sniffer several new families of formats. It is a major release because it drops old Rubies and changes the canonical type returned for some inputs — details below so you can scan for the ones that affect you.

Breaking changes

  • Ruby 3.3 or newer is required (was 2.3). (#170)
  • Canonical type changes — the same bytes or filename may now return a different (more standard) type:
    • Aliased types now resolve to their canonical form on output (#171): audio/x-aac → audio/aac, audio/x-flac → audio/flac, audio/vnd.wave → audio/x-wav, text/x-yaml → application/yaml, application/x-debian-package → application/vnd.debian.binary-package, application/x-xliff+xml → application/xliff+xml. Lookups by any alias still work.
    • image/bmp;format=compressed is now plain image/bmp (#148).
    • .cr2 files are image/x-canon-cr2 instead of image/x-raw-canon (#172).
    • OpenXPS (.oxps) is split from XPS (#171).
    • The XML/HTML detection work below can relabel documents that previously fell back to application/xml, text/html, or application/octet-stream.

Security & hardening

A review of the detection pipeline landed as a series of PRs (#157–#164):

  • MIME data is validated before table generation, and the generated tables are checked in CI (#157, rake tables:check).
  • HTML and XHTML magic matches only bounded opening-tag patterns instead of unanchored scans (#158).
  • IO edge cases — partial reads, unseekable and unrewindable IOs, non-string input — are handled explicitly while sniffing (#159).
  • Declared MIME types are validated rather than trusted (#160).
  • Marcel's detection boundary — what sniffing does and does not promise — is documented in the README (#161).
  • CI runs with locked dependencies (#163), and the release workflow builds and publishes the same verified gem artifact with sha256 recorded and sigstore attestation (#164), with the gem's contents verified by script (#162, #173).

New detection

  • OOXML documents (.docx/.xlsx/.pptx and their macro-enabled variants) are identified from the ZIP central directory, regardless of which application produced them (#169).
  • XML vocabularies are identified by their root element, the way Apache Tika does: RSS, Atom, KML, Apple property lists, XSLT, ONIX, iWork flat XML, Office 2003 XML, and friends (#173 and #175, fixing #48).
  • Canon CR2 raw images (#172).
  • RAR archives (#155, @tristan-f-r) and Google Sheets shortcuts (#156, @tristan-f-r).
  • AVIF image sequences (avis brand) (#151, @mm12) and audio-only .m4a files (#147).
  • HTML detection also checks the end of the document for closing tags (#146).

Performance

  • The common-type fast path is restored with subtype-aware ordering, so common formats like JPEG are identified in ~25 IO reads instead of hundreds, without preempting subtype matchers (#172, #174).
  • Sniffing uses IO#seek where possible instead of reading and discarding, including for regexp magic (#144, #145).

Upgrading

Most applications need no changes beyond Ruby ≥ 3.3. If you compare Marcel's output against stored MIME types, re-check the canonical type changes above — lookups by the old aliases still resolve, but Marcel now returns the canonical name.

- Ruby
Published by jeremy 3 months ago

https://github.com/rails/marcel - v1.2.1

  • Revert BMP images type to just image/bmp instead of image/bmp;format=compressed.
    The later is more precise, but cause backward compatibility issues with Active Storage.

Full Changelog: https://github.com/rails/marcel/compare/v1.2.0...v1.2.1

- Ruby
Published by byroot 6 months ago

https://github.com/rails/marcel - v1.2.0

What's Changed

New Contributors

Full Changelog: https://github.com/rails/marcel/compare/v1.1.1...v1.2.0

- Ruby
Published by byroot 6 months ago

https://github.com/rails/marcel - v1.1.1

What's Changed

New Contributors

Full Changelog: https://github.com/rails/marcel/compare/v1.1.0...v1.1.1

- Ruby
Published by byroot 6 months ago

https://github.com/rails/marcel - v1.1.0

What's Changed

New Contributors

Full Changelog: https://github.com/rails/marcel/compare/v1.0.4...v1.1.0

- Ruby
Published by rafaelfranca about 1 year ago

https://github.com/rails/marcel - v1.0.4

What's Changed

Full Changelog: https://github.com/rails/marcel/compare/v1.0.3...v1.0.4

- Ruby
Published by jeremy over 2 years ago

https://github.com/rails/marcel - v1.0.3

What's Changed

New Contributors

Full Changelog: https://github.com/rails/marcel/compare/v1.0.2...v1.0.3

- Ruby
Published by jeremy over 2 years ago

https://github.com/rails/marcel - v1.0.2

  • Include Apache license in gem release. (a525d5b38f287ca0511c8eb26e657a1d46686e5f)
  • Prefer audio/x-wav for WAV audio files. (#45)
  • Prefer application/x-x509-ca-cert for Privacy-Enhanced Mail certificates. (#46)
  • Prefer audio/flac for FLAC audio files. (#47)
  • Prefer audio/aac for Advanced Audio Coding audio files. (#49)
  • Prefer application/vnd.ms-access for Microsodt Access DB files. (#50)
  • Support text/x-scss and text/x-sass stylesheets. (#52)
  • Support encrypted Microsoft Access DB files. (#53)
  • Prefer application/x-ole-storage for Microsoft Office files. (#54)
  • Prefer text/markdown for Markdown files. (#55)
  • Prefer audio/mpc for Musepack audio files. (#56)
  • Support audio/webm audio files. (#58)
  • Support image/avif images files. (#63)

- Ruby
Published by rafaelfranca about 5 years ago

https://github.com/rails/marcel - v1.0.1

  • Fixes identifying OpenDocument files by magic. 1.0.0 imprecisely identified them as application/zip. (#38)
  • Fixes identifying .docx, .pptx, and .xlsx files exported from Google Sheets by magic. (#36)
  • Identifies vCard files as text/vcard rather than text/x-vcard. (27fac74bd69663495410339bfd40ea8581ba669b)
  • Identifies .otf, .woff, and .woff2 files aș font/otf, font/woff, and font/woff2, respectively. (#37)

- Ruby
Published by georgeclaghorn over 5 years ago

https://github.com/rails/marcel - v1.0.0

The mimemagic dependency—which relies on GPL-licensed mime type data from freedesktop.org’s shared-mime-info project—is removed. Marcel now directly uses mime type data adapted from the Apache Tika project, distributed under the Apache License.

- Ruby
Published by georgeclaghorn over 5 years ago

https://github.com/rails/marcel - v0.3.3

  • Shush Ruby interpreter warnings (#6)
  • Exclude test files from published gem to shrink it from 7.5 MB to 7.5 KB (#11)

- Ruby
Published by georgeclaghorn about 8 years ago

https://github.com/rails/marcel - v0.3.2

Emit the vnd/ms-powerpoint content type for *.ppt files. (#3)

- Ruby
Published by georgeclaghorn over 8 years ago

https://github.com/rails/marcel - v0.3.1

Support Ruby 2.2 for Rails compatibility.

- Ruby
Published by georgeclaghorn almost 9 years ago