https://github.com/rails/marcel
Find the mime type of files, examining file, filename and declared type
https://github.com/rails/marcel
Keywords from Contributors
activerecord activejob mvc rubygem rack rspec multithreading feature-flag ruby-gem background-jobs
Last synced: about 17 hours ago
JSON representation
Repository metadata
Find the mime type of files, examining file, filename and declared type
- Host: GitHub
- URL: https://github.com/rails/marcel
- Owner: rails
- License: apache-2.0
- Created: 2017-02-22T20:02:09.000Z (over 9 years ago)
- Default Branch: main
- Last Pushed: 2026-10-15T14:44:30.000Z (about 1 month ago)
- Last Synced: 2026-10-18T17:42:51.022Z (about 1 month ago)
- Language: Ruby
- Size: 5.62 MB
- Stars: 466
- Watchers: 24
- Forks: 81
- Open Issues: 5
- Releases: 14
-
Metadata Files:
- Readme: README.md
- License: MIT-LICENSE
- Security: SECURITY.md
README.md
Marcel
Marcel chooses the most appropriate content type for a file by inspecting its contents, the declared MIME type (perhaps passed as a Content-Type header), and the file extension.
Marcel checks, in order:
- The "magic bytes" sniffed from the file contents.
- The declared type, typically provided in a Content-Type header on an uploaded file, if it is a valid single media type other than the
application/octet-streamdefault. - The filename extension.
- Conservative fallback to the indeterminate
application/octet-streamdefault.
At each step, the most specific MIME subtype is selected. This allows the declared type and file extension to refine the parent type sniffed from the file contents, but not conflict with it. For example, if "file.csv" has declared type text/plain, text/csv is returned since it's a more specific subtype of text/plain. Similarly, Adobe Illustrator files are PDFs internally, so magic byte sniffing indicates application/pdf which is refined to application/illustrator by the ai file extension. But a PDF named "image.png" will still be detected as application/pdf since image/png is not a subtype. Specificity is based on MIME taxonomy, not proof that a file conforms to the selected type; see Security considerations below.
Declared types may include parameters and surrounding HTTP whitespace. As a deliberate compatibility recovery, Marcel tolerates a single trailing semicolon. It ignores other malformed and comma-separated declared types rather than choosing one value from a list.
Security considerations
Marcel is a best-effort file type labeler, not a file validator or a security boundary. The declared type and filename are caller-provided hints. A syntactically valid but unregistered declared type may be returned as-is when content magic does not conflict with it.
When content magic identifies a ZIP-based container, Marcel reads the archive's central directory listing — a bounded read from the end of seekable content — to distinguish Office document families and their macro-enabled variants by catalogued part names such as xl/vbaProject.bin. It reads only member names, not member contents, and does not verify format conformance or archive contents. A non-macro label is not proof that macros are absent: unseekable or truncated content, malformed archives, and formats catalogued under other names skip this refinement, and hints may still refine the generic container type. Likewise, Marcel::Magic#text? and #image? describe MIME taxonomy, not whether content is safe.
When content magic identifies generic XML, Marcel scans a bounded 64 KiB prefix for the document's root element and refines the label by the root's namespace and local name, following Apache Tika's root-element rules: feeds, KML, property lists, XSLT, XHTML and Office 2003 XML among others. The scan resolves no entities, DTD declarations, or external resources; as with Tika's namespace-aware SAX extraction, a prolog or root start-tag that is not well-formed at the token level — including namespace rules, forbidden characters, and bytes invalid in the declared encoding — refines nothing. Like Tika, it labels un-namespaced <body>, <p>, <script>, <frameset>, <iframe> and <link> roots as text/html. Prologs it cannot read, roots beyond the prefix, and unknown roots keep the generic application/xml label, which is therefore not proof that the content is inert markup.
Do not use Marcel's result by itself to decide whether content is safe to execute, parse with privileged features, or render inline. Apply the controls required by the consuming parser or renderer independently of the detected label.
Magic rules inspect bounded samples, so results can differ when a caller provides only a prefix. HTML detection recognizes document-opening <html> and <!DOCTYPE html> markers, not every fragment that can contain active markup. A result other than text/html is therefore not proof that the bytes are safe markup.
Serve untrusted uploads as attachments, restrict inline rendering to a small vetted allowlist, or isolate them on a separate untrusted origin. Also set an explicit Content-Type from a trusted source and X-Content-Type-Options: nosniff; those headers do not make correctly labelled active content safe to render inline. Rails Active Storage applies separate content-type and disposition controls when serving blobs. Call Marcel::Magic.by_magic directly when caller-provided filename and declared-type hints need to be evaluated separately from content magic.
Callers should apply their normal request and metadata size limits before invoking Marcel. Declared MIME types larger than 8 KiB are ignored. Content IOs must support rewind; buffer pipes and sockets before detection.
Usage
# Magic bytes sniffing alone
Marcel::MimeType.for Pathname.new("example.gif")
# => "image/gif"
File.open "example.gif" do |file|
Marcel::MimeType.for file
end
# => "image/gif"
# Magic bytes with filename fallback
Marcel::MimeType.for Pathname.new("unrecognisable-data"), name: "example.pdf"
# => "application/pdf"
# File extension alone
Marcel::MimeType.for extension: ".pdf"
# => "application/pdf"
# Magic bytes, declared type, and filename together
Marcel::MimeType.for Pathname.new("unrecognisable-data"), name: "example", declared_type: "image/png"
# => "image/png"
# Conservative fallback to application/octet-stream
Marcel::MimeType.for StringIO.new(File.read "unrecognisable-data")
# => "application/octet-stream"
Extending
Custom file types may be added with Marcel::MimeType.extend:
Marcel::MimeType.extend "text/custom", extensions: %w( customtxt )
Marcel::MimeType.for name: "file.customtxt"
# => "text/custom"
Registration mutates a process-global registry. Add custom types during single-threaded application boot, before concurrent detection begins. Extension collisions replace the existing mapping, and removing the custom type does not restore a mapping it replaced.
Motivation
Marcel was extracted from Basecamp's file detection heuristics. The aim is to provide sensible, conservative, "do what I expect" results for typical file handling. Test fixtures have been added for many common file types, including those typically encountered by Basecamp.
Contributing
Marcel generates MIME lookup tables with bundle exec rake update. MIME types are seeded from data found in data/*.xml. The Apache Tika download and committed data/tika.xml are verified against the commit and checksum in script/download_tika_data.rb; review and update both values when refreshing it. Custom MIMEs may be added to data/custom.xml, while code-defined overrides live in the definitions files under lib/marcel.
Marcel follows the same contributing guidelines as rails/rails.
Testing
The main test fixture files are split into two folders, those that can be recognised by magic bytes, and those that can only be recognised by name. Even though strictly unnecessary, the fixtures in both folders should all be valid files of the type they represent.
License
Marcel itself is released under the terms of the MIT License. See the MIT-LICENSE file for details.
Portions of Marcel are adapted from the mimemagic gem, released under the terms of the MIT License.
Marcel's magic signature data is adapted from Apache Tika, released under the terms of the Apache License. See the APACHE-LICENSE file for details.
Owner metadata
- Name: Ruby on Rails
- Login: rails
- Email:
- Kind: organization
- Description:
- Website: https://rubyonrails.org/
- Location:
- Twitter:
- Company:
- Icon url: https://avatars.githubusercontent.com/u/4223?v=4
- Repositories: 130
- Last ynced at: 2026-11-13T21:43:58.389Z
- Profile URL: https://github.com/rails
GitHub Events
Total
- Create event: 7
- Delete event: 2
- Fork event: 5
- Issue comment event: 16
- Issues event: 13
- Pull request event: 10
- Pull request review comment event: 6
- Pull request review event: 7
- Push event: 18
- Watch event: 37
Last Year
- Create event: 4
- Delete event: 1
- Fork event: 1
- Issue comment event: 2
- Issues event: 1
- Pull request event: 3
- Pull request review event: 1
- Push event: 12
- Watch event: 7
Committers metadata
Last synced: 1 day ago
Total Commits: 206
Total Committers: 41
Avg Commits per committer: 5.024
Development Distribution Score (DDS): 0.738
Commits in past year: 52
Committers in past year: 11
Avg Commits per committer in past year: 4.727
Development Distribution Score (DDS) in past year: 0.596
| Name | Commits | |
|---|---|---|
| Tom Ward | t****m@p****t | 54 |
| Jeremy Daer | j****y@r****g | 38 |
| Jean Boussier | j****r@g****m | 22 |
| George Claghorn | g****e@b****m | 16 |
| Gannon McGibbon | g****n@g****m | 15 |
| Alexander ADAM | a****m@v****o | 9 |
| Rafael Mendonça França | r****l@r****g | 5 |
| Jun Aruga | j****a@r****m | 3 |
| Tristan F.-R. | p****f@g****m | 3 |
| dependabot[bot] | 4****] | 3 |
| Eddie Lebow | E****w@p****m | 2 |
| Olle Jonsson | o****n@g****m | 2 |
| Chris Oliver | e****3@g****m | 2 |
| Catting | 5****2 | 2 |
| Brian Kephart | b****t@g****m | 2 |
| Andy Nguyen | a****n@s****m | 2 |
| Andreas Lillebø | g****t@a****m | 2 |
| Alberto Fernandez-Capel | a****o@h****m | 1 |
| Alexander Gitter | c****t@a****t | 1 |
| Andrew Nutter-Upham | a****u@g****m | 1 |
| Ben Koshy | b****y@h****m | 1 |
| yuuji.yaginuma | y****a@g****m | 1 |
| wonda-tea-coffee | l****t@g****m | 1 |
| siutin | o****o@g****m | 1 |
| mark-young-atg | 1****g | 1 |
| eileencodes | e****s@g****m | 1 |
| Vipul A M | v****d@g****m | 1 |
| Valeriy Mironov | v****m@g****m | 1 |
| Stano Bo | sb@j****t | 1 |
| Mth0158 | m****y@g****m | 1 |
| and 11 more... | ||
Committer domains:
- rubyonrails.org: 2
- frances.co: 1
- ursm.jp: 1
- youngcapital.nl: 1
- juniq.net: 1
- agitter.net: 1
- hey.com: 1
- andreaslillebo.com: 1
- shopify.com: 1
- phishme.com: 1
- redhat.com: 1
- vade.io: 1
- basecamp.com: 1
- popdog.net: 1
Issue and Pull Request metadata
Last synced: 1 day ago
Total issues: 63
Total pull requests: 137
Average time to close issues: about 1 year
Average time to close pull requests: 4 months
Total issue authors: 58
Total pull request authors: 53
Average comments per issue: 2.0
Average comments per pull request: 0.69
Merged pull request: 85
Bot issues: 0
Bot pull requests: 15
Past year issues: 8
Past year pull requests: 44
Past year average time to close issues: 4 days
Past year average time to close pull requests: 10 days
Past year issue authors: 4
Past year pull request authors: 10
Past year average comments per issue: 0.0
Past year average comments per pull request: 0.16
Past year merged pull request: 18
Past year bot issues: 0
Past year bot pull requests: 12
Top Issue Authors
- OskarEichler (5)
- tomafro (2)
- johnnyshields (1)
- ghost (1)
- markedmondson (1)
- IBazylchuk (1)
- genezys (1)
- AlfonsoUceda (1)
- nvanoorschot (1)
- timfrazee (1)
- ClearlyClaire (1)
- gr8bit (1)
- pcantrell (1)
- ron-shinall (1)
- andyvanee (1)
Top Pull Request Authors
- jeremy (21)
- dependabot[bot] (15)
- gmcgibbon (11)
- alexanderadam (10)
- byroot (5)
- junaruga (4)
- casperisfine (3)
- tristan-f-r (3)
- georgeclaghorn (3)
- hahmed (2)
- olleolleolle (2)
- Andy-H-Nguyen (2)
- elebow (2)
- vipulnsward (2)
- rafaelfranca (2)
Top Issue Labels
Top Pull Request Labels
- dependencies (15)
- github_actions (9)
- ruby (3)
Package metadata
- Total packages: 6
-
Total downloads:
- rubygems: 1,016,908,544 total
- Total docker downloads: 5,984,095,580
- Total dependent packages: 48 (may contain duplicates)
- Total dependent repositories: 290,032 (may contain duplicates)
- Total versions: 48
- Total maintainers: 12
gem.coop: marcel
Simple mime type detection using magic numbers, filenames, and extensions
- Homepage: https://github.com/rails/marcel
- Documentation: http://www.rubydoc.info/gems/marcel/
- Licenses: MIT,Apache-2.0
- Latest release: 2.1.0 (published 3 months ago)
- Last Synced: 2026-11-16T15:00:34.919Z (1 day ago)
- Versions: 15
- Dependent Packages: 0
- Dependent Repositories: 0
- Downloads: 508,582,263 Total
- Docker Downloads: 2,992,047,790
-
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 0.026%
- Docker downloads count: 0.045%
- Downloads: 0.058%
- Maintainers (12)
debian-13: ruby-marcel
- Homepage: https://github.com/rails/marcel
- Documentation: https://packages.debian.org/trixie/ruby-marcel
- Licenses: apache-2.0
- Latest release: 1.0.4+dfsg-2 (published 9 months ago)
- Last Synced: 2026-07-28T16:02:41.262Z (4 months ago)
- Versions: 1
- Dependent Packages: 0
- Dependent Repositories: 0
-
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 0.336%
- Forks count: 0.605%
- Stargazers count: 0.738%
rubygems.org: marcel
Simple mime type detection using magic numbers, filenames, and extensions
- Homepage: https://github.com/rails/marcel
- Documentation: http://www.rubydoc.info/gems/marcel/
- Licenses: MIT,Apache-2.0
- Latest release: 2.1.0 (published 3 months ago)
- Last Synced: 2026-11-16T03:03:26.938Z (1 day ago)
- Versions: 15
- Dependent Packages: 48
- Dependent Repositories: 290,032
- Downloads: 508,326,281 Total
- Docker Downloads: 2,992,047,790
-
Rankings:
- Downloads: 0.069%
- Docker downloads count: 0.069%
- Dependent repos count: 0.084%
- Dependent packages count: 0.611%
- Average: 1.19%
- Stargazers count: 2.948%
- Forks count: 3.361%
- Maintainers (12)
proxy.golang.org: github.com/rails/marcel
- Homepage:
- Documentation: https://pkg.go.dev/github.com/rails/marcel#section-documentation
- Licenses: apache-2.0
- Latest release: v2.1.0+incompatible (published 3 months ago)
- Last Synced: 2026-11-16T03:09:53.745Z (1 day ago)
- Versions: 14
- Dependent Packages: 0
- Dependent Repositories: 0
-
Rankings:
- Stargazers count: 2.964%
- Forks count: 3.095%
- Average: 6.609%
- Dependent packages count: 9.576%
- Dependent repos count: 10.802%
guix: ruby-marcel
MIME type detection using magic numbers, filenames and extensions
- Homepage: https://github.com/rails/marcel
- Documentation: https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/rails.scm#n1036
- Licenses: expat
- Latest release: 1.0.4 (published 9 months ago)
- Last Synced: 2026-04-27T16:22:48.332Z (7 months ago)
- Versions: 1
- Dependent Packages: 0
- Dependent Repositories: 0
-
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%
gentoo-portage: dev-ruby/marcel
Simple mime type detection using magic numbers, filenames, and extensions
- Homepage: https://github.com/rails/marcel
- Documentation: https://packages.gentoo.org/packages/dev-ruby/marcel
- Licenses: Apache-2.0 MIT
- Latest release: 1.1.0 (published 7 months ago)
- Last Synced: 2026-05-27T02:47:06.156Z (6 months ago)
- Versions: 2
- Dependent Packages: 0
- Dependent Repositories: 0
-
Rankings:
- Dependent repos count: 0.0%
- Dependent packages count: 0.0%
- Average: 100%
Dependencies
- bundler >= 1.7 development
- byebug ~> 10.0.2 development
- minitest ~> 5.11 development
- nokogiri >= 1.9.1 development
- rack ~> 2.0 development
- rake ~> 13.0 development
- actions/checkout v5 composite
- ruby/setup-ruby v1 composite
- rubygems/release-gem v1 composite
- actions/checkout v3 composite
- ruby/setup-ruby v1 composite
- bundler 4.0.6
- drb 2.2.3
- marcel 2.1.0
- minitest 6.0.6
- prism 1.9.0
- rack 3.2.7
- rake 13.4.2
- bundler 4.0.6
- logger 1.7.0
- net-http 0.9.1
- protobug 0.1.0
- protobug_googleapis_field_behavior_protos 0.1.0
- protobug_sigstore_protos 0.1.0
- protobug_well_known_protos 0.1.0
- sigstore 0.2.3
- sigstore-cli 0.2.3
- thor 1.5.0
- uri 1.1.1
- sigstore-cli = 0.2.3
- bundler 4.0.6
- debug 1.11.1
- drb 2.2.3
- erb 6.0.7
- io-console 0.9.2
- irb 1.18.0
- logger 1.7.0
- marcel 2.1.0
- minitest 6.0.6
- nokogiri 1.19.4
- pp 0.6.4
- prettyprint 0.2.0
- prism 1.9.0
- racc 1.8.1
- rack 3.2.7
- rake 13.4.2
- rbs 4.1.3
- rdoc 8.0.0
- reline 0.7.0
- tsort 0.2.0
Score: 32.53779143241314