A summary of data about the Ruby ecosystem.

https://github.com/rails/marcel

Find the mime type of files, examining file, filename and declared type
https://github.com/rails/marcel

Keywords from Contributors

activerecord activejob mvc rubygem rack rspec multithreading feature-flag ruby-gem background-jobs

Last synced: about 17 hours ago
JSON representation

Repository metadata

Find the mime type of files, examining file, filename and declared type

README.md

Marcel

Marcel chooses the most appropriate content type for a file by inspecting its contents, the declared MIME type (perhaps passed as a Content-Type header), and the file extension.

Marcel checks, in order:

  1. The "magic bytes" sniffed from the file contents.
  2. The declared type, typically provided in a Content-Type header on an uploaded file, if it is a valid single media type other than the application/octet-stream default.
  3. The filename extension.
  4. Conservative fallback to the indeterminate application/octet-stream default.

At each step, the most specific MIME subtype is selected. This allows the declared type and file extension to refine the parent type sniffed from the file contents, but not conflict with it. For example, if "file.csv" has declared type text/plain, text/csv is returned since it's a more specific subtype of text/plain. Similarly, Adobe Illustrator files are PDFs internally, so magic byte sniffing indicates application/pdf which is refined to application/illustrator by the ai file extension. But a PDF named "image.png" will still be detected as application/pdf since image/png is not a subtype. Specificity is based on MIME taxonomy, not proof that a file conforms to the selected type; see Security considerations below.

Declared types may include parameters and surrounding HTTP whitespace. As a deliberate compatibility recovery, Marcel tolerates a single trailing semicolon. It ignores other malformed and comma-separated declared types rather than choosing one value from a list.

Security considerations

Marcel is a best-effort file type labeler, not a file validator or a security boundary. The declared type and filename are caller-provided hints. A syntactically valid but unregistered declared type may be returned as-is when content magic does not conflict with it.

When content magic identifies a ZIP-based container, Marcel reads the archive's central directory listing — a bounded read from the end of seekable content — to distinguish Office document families and their macro-enabled variants by catalogued part names such as xl/vbaProject.bin. It reads only member names, not member contents, and does not verify format conformance or archive contents. A non-macro label is not proof that macros are absent: unseekable or truncated content, malformed archives, and formats catalogued under other names skip this refinement, and hints may still refine the generic container type. Likewise, Marcel::Magic#text? and #image? describe MIME taxonomy, not whether content is safe.

When content magic identifies generic XML, Marcel scans a bounded 64 KiB prefix for the document's root element and refines the label by the root's namespace and local name, following Apache Tika's root-element rules: feeds, KML, property lists, XSLT, XHTML and Office 2003 XML among others. The scan resolves no entities, DTD declarations, or external resources; as with Tika's namespace-aware SAX extraction, a prolog or root start-tag that is not well-formed at the token level — including namespace rules, forbidden characters, and bytes invalid in the declared encoding — refines nothing. Like Tika, it labels un-namespaced <body>, <p>, <script>, <frameset>, <iframe> and <link> roots as text/html. Prologs it cannot read, roots beyond the prefix, and unknown roots keep the generic application/xml label, which is therefore not proof that the content is inert markup.

Do not use Marcel's result by itself to decide whether content is safe to execute, parse with privileged features, or render inline. Apply the controls required by the consuming parser or renderer independently of the detected label.

Magic rules inspect bounded samples, so results can differ when a caller provides only a prefix. HTML detection recognizes document-opening <html> and <!DOCTYPE html> markers, not every fragment that can contain active markup. A result other than text/html is therefore not proof that the bytes are safe markup.

Serve untrusted uploads as attachments, restrict inline rendering to a small vetted allowlist, or isolate them on a separate untrusted origin. Also set an explicit Content-Type from a trusted source and X-Content-Type-Options: nosniff; those headers do not make correctly labelled active content safe to render inline. Rails Active Storage applies separate content-type and disposition controls when serving blobs. Call Marcel::Magic.by_magic directly when caller-provided filename and declared-type hints need to be evaluated separately from content magic.

Callers should apply their normal request and metadata size limits before invoking Marcel. Declared MIME types larger than 8 KiB are ignored. Content IOs must support rewind; buffer pipes and sockets before detection.

Usage

# Magic bytes sniffing alone
Marcel::MimeType.for Pathname.new("example.gif")
#  => "image/gif"

File.open "example.gif" do |file|
  Marcel::MimeType.for file
end
#  => "image/gif"

# Magic bytes with filename fallback
Marcel::MimeType.for Pathname.new("unrecognisable-data"), name: "example.pdf"
#  => "application/pdf"

# File extension alone
Marcel::MimeType.for extension: ".pdf"
#  => "application/pdf"

# Magic bytes, declared type, and filename together
Marcel::MimeType.for Pathname.new("unrecognisable-data"), name: "example", declared_type: "image/png"
#  => "image/png"

# Conservative fallback to application/octet-stream
Marcel::MimeType.for StringIO.new(File.read "unrecognisable-data")
#  => "application/octet-stream"

Extending

Custom file types may be added with Marcel::MimeType.extend:

Marcel::MimeType.extend "text/custom", extensions: %w( customtxt )
Marcel::MimeType.for name: "file.customtxt"
#  => "text/custom"

Registration mutates a process-global registry. Add custom types during single-threaded application boot, before concurrent detection begins. Extension collisions replace the existing mapping, and removing the custom type does not restore a mapping it replaced.

Motivation

Marcel was extracted from Basecamp's file detection heuristics. The aim is to provide sensible, conservative, "do what I expect" results for typical file handling. Test fixtures have been added for many common file types, including those typically encountered by Basecamp.

Contributing

Marcel generates MIME lookup tables with bundle exec rake update. MIME types are seeded from data found in data/*.xml. The Apache Tika download and committed data/tika.xml are verified against the commit and checksum in script/download_tika_data.rb; review and update both values when refreshing it. Custom MIMEs may be added to data/custom.xml, while code-defined overrides live in the definitions files under lib/marcel.

Marcel follows the same contributing guidelines as rails/rails.

Testing

The main test fixture files are split into two folders, those that can be recognised by magic bytes, and those that can only be recognised by name. Even though strictly unnecessary, the fixtures in both folders should all be valid files of the type they represent.

License

Marcel itself is released under the terms of the MIT License. See the MIT-LICENSE file for details.

Portions of Marcel are adapted from the mimemagic gem, released under the terms of the MIT License.

Marcel's magic signature data is adapted from Apache Tika, released under the terms of the Apache License. See the APACHE-LICENSE file for details.


Owner metadata


GitHub Events

Total
Last Year

Committers metadata

Last synced: 1 day ago

Total Commits: 206
Total Committers: 41
Avg Commits per committer: 5.024
Development Distribution Score (DDS): 0.738

Commits in past year: 52
Committers in past year: 11
Avg Commits per committer in past year: 4.727
Development Distribution Score (DDS) in past year: 0.596

Name Email Commits
Tom Ward t****m@p****t 54
Jeremy Daer j****y@r****g 38
Jean Boussier j****r@g****m 22
George Claghorn g****e@b****m 16
Gannon McGibbon g****n@g****m 15
Alexander ADAM a****m@v****o 9
Rafael Mendonça França r****l@r****g 5
Jun Aruga j****a@r****m 3
Tristan F.-R. p****f@g****m 3
dependabot[bot] 4****] 3
Eddie Lebow E****w@p****m 2
Olle Jonsson o****n@g****m 2
Chris Oliver e****3@g****m 2
Catting 5****2 2
Brian Kephart b****t@g****m 2
Andy Nguyen a****n@s****m 2
Andreas Lillebø g****t@a****m 2
Alberto Fernandez-Capel a****o@h****m 1
Alexander Gitter c****t@a****t 1
Andrew Nutter-Upham a****u@g****m 1
Ben Koshy b****y@h****m 1
yuuji.yaginuma y****a@g****m 1
wonda-tea-coffee l****t@g****m 1
siutin o****o@g****m 1
mark-young-atg 1****g 1
eileencodes e****s@g****m 1
Vipul A M v****d@g****m 1
Valeriy Mironov v****m@g****m 1
Stano Bo sb@j****t 1
Mth0158 m****y@g****m 1
and 11 more...

Committer domains:


Issue and Pull Request metadata

Last synced: 1 day ago

Total issues: 63
Total pull requests: 137
Average time to close issues: about 1 year
Average time to close pull requests: 4 months
Total issue authors: 58
Total pull request authors: 53
Average comments per issue: 2.0
Average comments per pull request: 0.69
Merged pull request: 85
Bot issues: 0
Bot pull requests: 15

Past year issues: 8
Past year pull requests: 44
Past year average time to close issues: 4 days
Past year average time to close pull requests: 10 days
Past year issue authors: 4
Past year pull request authors: 10
Past year average comments per issue: 0.0
Past year average comments per pull request: 0.16
Past year merged pull request: 18
Past year bot issues: 0
Past year bot pull requests: 12

More stats: https://issues.ecosyste.ms/repositories/lookup?url=https://github.com/rails/marcel

Top Issue Authors

  • OskarEichler (5)
  • tomafro (2)
  • johnnyshields (1)
  • ghost (1)
  • markedmondson (1)
  • IBazylchuk (1)
  • genezys (1)
  • AlfonsoUceda (1)
  • nvanoorschot (1)
  • timfrazee (1)
  • ClearlyClaire (1)
  • gr8bit (1)
  • pcantrell (1)
  • ron-shinall (1)
  • andyvanee (1)

Top Pull Request Authors

  • jeremy (21)
  • dependabot[bot] (15)
  • gmcgibbon (11)
  • alexanderadam (10)
  • byroot (5)
  • junaruga (4)
  • casperisfine (3)
  • tristan-f-r (3)
  • georgeclaghorn (3)
  • hahmed (2)
  • olleolleolle (2)
  • Andy-H-Nguyen (2)
  • elebow (2)
  • vipulnsward (2)
  • rafaelfranca (2)

Top Issue Labels

Top Pull Request Labels

  • dependencies (15)
  • github_actions (9)
  • ruby (3)

Package metadata

gem.coop: marcel

Simple mime type detection using magic numbers, filenames, and extensions

debian-13: ruby-marcel

  • Homepage: https://github.com/rails/marcel
  • Documentation: https://packages.debian.org/trixie/ruby-marcel
  • Licenses: apache-2.0
  • Latest release: 1.0.4+dfsg-2 (published 9 months ago)
  • Last Synced: 2026-07-28T16:02:41.262Z (4 months ago)
  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Rankings:
    • Dependent repos count: 0.0%
    • Dependent packages count: 0.0%
    • Average: 0.336%
    • Forks count: 0.605%
    • Stargazers count: 0.738%
rubygems.org: marcel

Simple mime type detection using magic numbers, filenames, and extensions

proxy.golang.org: github.com/rails/marcel

  • Homepage:
  • Documentation: https://pkg.go.dev/github.com/rails/marcel#section-documentation
  • Licenses: apache-2.0
  • Latest release: v2.1.0+incompatible (published 3 months ago)
  • Last Synced: 2026-11-16T03:09:53.745Z (1 day ago)
  • Versions: 14
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Rankings:
    • Stargazers count: 2.964%
    • Forks count: 3.095%
    • Average: 6.609%
    • Dependent packages count: 9.576%
    • Dependent repos count: 10.802%
guix: ruby-marcel

MIME type detection using magic numbers, filenames and extensions

  • Homepage: https://github.com/rails/marcel
  • Documentation: https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/rails.scm#n1036
  • Licenses: expat
  • Latest release: 1.0.4 (published 9 months ago)
  • Last Synced: 2026-04-27T16:22:48.332Z (7 months ago)
  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Rankings:
    • Dependent repos count: 0.0%
    • Dependent packages count: 0.0%
    • Average: 100%
gentoo-portage: dev-ruby/marcel

Simple mime type detection using magic numbers, filenames, and extensions

  • Homepage: https://github.com/rails/marcel
  • Documentation: https://packages.gentoo.org/packages/dev-ruby/marcel
  • Licenses: Apache-2.0 MIT
  • Latest release: 1.1.0 (published 7 months ago)
  • Last Synced: 2026-05-27T02:47:06.156Z (6 months ago)
  • Versions: 2
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Rankings:
    • Dependent repos count: 0.0%
    • Dependent packages count: 0.0%
    • Average: 100%

Dependencies

marcel.gemspec rubygems
  • bundler >= 1.7 development
  • byebug ~> 10.0.2 development
  • minitest ~> 5.11 development
  • nokogiri >= 1.9.1 development
  • rack ~> 2.0 development
  • rake ~> 13.0 development
.github/workflows/release.yml actions
  • actions/checkout v5 composite
  • ruby/setup-ruby v1 composite
  • rubygems/release-gem v1 composite
.github/workflows/ci.yml actions
  • actions/checkout v3 composite
  • ruby/setup-ruby v1 composite
Gemfile rubygems
gemfiles/runtime/Gemfile.lock rubygems
  • bundler 4.0.6
  • drb 2.2.3
  • marcel 2.1.0
  • minitest 6.0.6
  • prism 1.9.0
  • rack 3.2.7
  • rake 13.4.2
gemfiles/runtime/Gemfile rubygems
gemfiles/release/Gemfile.lock rubygems
  • bundler 4.0.6
  • logger 1.7.0
  • net-http 0.9.1
  • protobug 0.1.0
  • protobug_googleapis_field_behavior_protos 0.1.0
  • protobug_sigstore_protos 0.1.0
  • protobug_well_known_protos 0.1.0
  • sigstore 0.2.3
  • sigstore-cli 0.2.3
  • thor 1.5.0
  • uri 1.1.1
gemfiles/release/Gemfile rubygems
  • sigstore-cli = 0.2.3
Gemfile.lock rubygems
  • bundler 4.0.6
  • debug 1.11.1
  • drb 2.2.3
  • erb 6.0.7
  • io-console 0.9.2
  • irb 1.18.0
  • logger 1.7.0
  • marcel 2.1.0
  • minitest 6.0.6
  • nokogiri 1.19.4
  • pp 0.6.4
  • prettyprint 0.2.0
  • prism 1.9.0
  • racc 1.8.1
  • rack 3.2.7
  • rake 13.4.2
  • rbs 4.1.3
  • rdoc 8.0.0
  • reline 0.7.0
  • tsort 0.2.0

Score: 32.53779143241314