Data

Tools

Indexes

Applications

Experiments

Ruby

A summary of data about the Ruby ecosystem.

https://github.com/whitequark/parser

A Ruby parser.
https://github.com/whitequark/parser

Keywords from Contributors

rubocop code-formatter static-code-analysis rubygems activerecord activejob mvc rspec rack static-analysis

Last synced: about 20 hours ago
JSON representation

Repository metadata

A Ruby parser.

README.md

Parser

Gem Version
Tests

Parser is a production-ready Ruby parser written in pure Ruby. It recognizes as
much or more code than Ripper, Melbourne, JRubyParser or ruby_parser, and
is vastly more convenient to use.

You can also use unparser to produce
equivalent source code from Parser's ASTs.

Sponsored by Evil Martians.
MacRuby and RubyMotion support sponsored by CodeClimate.

[!WARNING]
The parser gem is only compatible with the syntax of Ruby 3.3 and lower. For Ruby 3.4 and later, please use the Prism::Translation::Parser instead.
Starting in Ruby 3.4, Prism is the parser used in Ruby itself and can produce AST that is identical to the output of the parser gem. If you only need to parse Ruby 3.3 (or greater) and don't require compatibility with the parser gem AST, also consider using the native Prism AST.
See this GitHub issue for more details.
For a guide on how to use parser for older versions and prism for newer ones, please see this guide.

Installation

$ gem install parser

Usage

Load Parser (see the backwards compatibility section
below for explanation of emit_* calls):

require 'parser/current'
# opt-in to most recent AST format:
Parser::Builders::Default.emit_lambda              = true
Parser::Builders::Default.emit_procarg0            = true
Parser::Builders::Default.emit_encoding            = true
Parser::Builders::Default.emit_index               = true
Parser::Builders::Default.emit_arg_inside_procarg0 = true
Parser::Builders::Default.emit_forward_arg         = true
Parser::Builders::Default.emit_kwargs              = true
Parser::Builders::Default.emit_match_pattern       = true

Parse a chunk of code:

p Parser::CurrentRuby.parse("2 + 2")
# (send
#   (int 2) :+
#   (int 2))

Access the AST's source map:

p Parser::CurrentRuby.parse("2 + 2").loc
# #<Parser::Source::Map::Send:0x007fe5a1ac2388
#   @dot=nil,
#   @begin=nil,
#   @end=nil,
#   @selector=#<Source::Range (string) 2...3>,
#   @expression=#<Source::Range (string) 0...5>>

p Parser::CurrentRuby.parse("2 + 2").loc.selector.source
# "+"

Traverse the AST: see the documentation for gem ast.

Parse a chunk of code and display all diagnostics:

parser = Parser::CurrentRuby.new
parser.diagnostics.consumer = lambda do |diag|
  puts diag.render
end

buffer = Parser::Source::Buffer.new('(string)', source: "foo *bar")

p parser.parse(buffer)
# (string):1:5: warning: `*' interpreted as argument prefix
# foo *bar
#     ^
# (send nil :foo
#   (splat
#     (send nil :bar)))

If you reuse the same parser object for multiple #parse runs, you need to
#reset it.

You can also use the ruby-parse utility (it's bundled with the gem) to play
with Parser:

$ ruby-parse -L -e "2+2"
(send
  (int 2) :+
  (int 2))
2+2
 ~ selector
~~~ expression
(int 2)
2+2
~ expression
(int 2)
2+2

$ ruby-parse -E -e "2+2"
2+2
^ tINTEGER 2                                    expr_end     [0 <= cond] [0 <= cmdarg]
2+2
 ^ tPLUS "+"                                    expr_beg     [0 <= cond] [0 <= cmdarg]
2+2
  ^ tINTEGER 2                                  expr_end     [0 <= cond] [0 <= cmdarg]
2+2
  ^ false "$eof"                                expr_end     [0 <= cond] [0 <= cmdarg]
(send
  (int 2) :+
  (int 2))

Features

  • Precise source location reporting.
  • Documented AST format which is convenient to work with.
  • A simple interface and a powerful, tweakable one.
  • Parses 1.8, 1.9, 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 3.0, 3.1, 3.2, and 3.3 syntax with backwards-compatible
    AST formats.
  • Parses MacRuby and RubyMotion syntax extensions.
  • Rewriting support.
  • Parsing error recovery.
  • Improved clang-like diagnostic messages with location information.
  • Written in pure Ruby, runs on MRI >=2.0.0, JRuby and Rubinius (and historically, all versions of Ruby since 1.8)
  • Only one runtime dependency: the ast gem.
  • Insane Ruby lexer rewritten from scratch in Ragel.
  • 100% test coverage for Bison grammars (except error recovery).
  • Readable, commented source code.

Documentation

Documentation for Parser is available online.

Node names

Several Parser nodes seem to be confusing enough to warrant a dedicated README section.

(block)

The (block) node passes a Ruby block, that is, a closure, to a method call represented by its first child, a (send), (super) or (zsuper) node. To demonstrate:

$ ruby-parse -e 'foo { |x| x + 2 }'
(block
  (send nil :foo)
  (args
    (arg :x))
  (send
    (lvar :x) :+
    (int 2)))

(begin) and (kwbegin)

TL;DR: Unless you perform rewriting, treat (begin) and (kwbegin) as the same node type.

Both (begin) and (kwbegin) nodes represent compound statements, that is, several expressions which are executed sequentally and the value of the last one is the value of entire compound statement. They may take several forms in the source code:

  • foo; bar: without delimiters
  • (foo; bar): parenthesized
  • begin foo; bar; end: grouped with begin keyword
  • def x; foo; bar; end: grouped inside a method definition

and so on.

$ ruby-parse -e '(foo; bar)'
(begin
  (send nil :foo)
  (send nil :bar))
$ ruby-parse -e 'def x; foo; bar end'
(def :x
  (args)
  (begin
    (send nil :foo)
    (send nil :bar)))

Note that, despite its name, kwbegin node only has tangential relation to the begin keyword. Normally, Parser AST is semantic, that is, if two constructs look differently but behave identically, they get parsed to the same node. However, there exists a peculiar construct called post-loop in Ruby:

begin
  body
end while condition

This specific syntactic construct, that is, keyword begin..end block followed by a postfix while, behaves very unlike other similar constructs, e.g. (body) while condition. While the body itself is wrapped into a while-post node, Parser also supports rewriting, and in that context it is important to not accidentally convert one kind of loop into another.

$ ruby-parse -e 'begin foo end while cond'
(while-post
  (send nil :cond)
  (kwbegin
    (send nil :foo)))
$ ruby-parse -e 'foo while cond'
(while
  (send nil :cond)
  (send nil :foo))
$ ruby-parse -e '(foo) while cond'
(while
  (send nil :cond)
  (begin
    (send nil :foo)))

(Parser also needs the (kwbegin) node type internally, and it is highly problematic to map it back to (begin).)

Backwards compatibility

Parser does not use semantic versioning. Parser versions are structured as x.y.z.t,
where x.y.z indicates the most recent supported Ruby release (support for every
Ruby release that is chronologically earlier is implied), and t is a monotonically
increasing number.

The public API of Parser as well as the AST format (as listed in the documentation)
are considered stable forever, although support for old Ruby versions may be removed
at some point.

Sometimes it is necessary to modify the format of AST nodes that are already being emitted
in a way that would break existing applications. To avoid such breakage, applications
must opt-in to these modifications; without explicit opt-in, Parser will continue to emit
the old AST node format. The most recent set of opt-ins is specified in
the usage section of this README.

Compatibility with Ruby MRI

Unfortunately, Ruby MRI often changes syntax in patchlevel versions. This has happened, at least, for every release since 1.9; for example, commits c5013452 and 04bb9d6b were backported all the way from HEAD to 1.9. Moreover, there is no simple way to track these changes.

This policy makes it all but impossible to make Parser precisely compatible with the Ruby MRI parser. Indeed, at September 2014, it would be necessary to maintain and update ten different parsers together with their lexer quirks in order to be able to emulate any given released Ruby MRI version.

As a result, Parser chooses a different path: the parser/rubyXY parsers recognize the syntax of the latest minor version of Ruby MRI X.Y at the time of the gem release.

Compatibility with MacRuby and RubyMotion

Parser implements the MacRuby 0.12 and RubyMotion mid-2015 parsers precisely. However, the lexers of these have been forked off Ruby MRI and independently maintained for some time, and because of that, Parser may accept some code that these upstream implementations are unable to parse.

Known issues

Adding support for the following Ruby MRI features in Parser would needlessly complicate it, and as they all are very specific and rarely occurring corner cases, this is not done.

Parser has been extensively tested; in particular, it parses almost entire Rubygems corpus. For every issue, a breakdown of affected gems is offered.

Void value expressions

Ruby MRI prohibits so-called "void value expressions". For a description
of what a void value expression is, see this
gist and this Parser
issue.

It is unknown whether any gems are affected by this issue.

Syntax check of block exits

Similar to "void value expression" checks Ruby MRI also checks for correct
usage of break, next and redo, if it's used outside of a {break,next,redo}-able
context Ruby returns a syntax error starting from 3.3.0. parser gem simply doesn't
run this type of checks.

It is unknown whether any gems are affected by this issue.

Invalid characters inside comments and literals

Ruby MRI permits arbitrary non-7-bit byte sequences to appear in comments, as well as in string or symbol literals in form of escape sequences, regardless of source encoding. Parser requires all source code, including the expanded escape sequences, to consist of valid byte sequences in the source encoding that are convertible to UTF-8.

As of 2013-07-25, there are about 180 affected gems.

\u escape in 1.8 mode

Ruby MRI 1.8 permits to specify a bare \u escape sequence in a string; it treats it like u. Ruby MRI 1.9 and later treat \u as a prefix for Unicode escape sequence and do not allow it to appear bare. Parser follows 1.9+ behavior.

As of 2013-07-25, affected gems are: activerdf, activerdf_net7, fastreader, gkellog-reddy.

Dollar-dash

(This one is so obscure I couldn't even think of a saner name for this issue.) Pre-2.1 Ruby allows
to specify a global variable named $-. Ruby 2.1 and later treat it as a syntax error. Parser
follows 2.1 behavior.

No known code is affected by this issue.

EOF characters after embedded documents before 2.7

Code like "=begin\n""=end\0" is invalid for all versions of Ruby before 2.7. Ruby 2.7 and later parses it
normally. Parser follows 2.7 behavior.

It is unknown whether any gems are affected by this issue.

Contributors

Acknowledgements

The lexer testsuite is derived from
ruby_parser.

The Bison parser rules are derived from Ruby MRI
parse.y.

Contributing

  1. Make sure you have Ragel ~> 6.7 installed
  2. Fork it
  3. Create your feature branch (git checkout -b my-new-feature)
  4. Commit your changes (git commit -am 'Add some feature')
  5. Push to the branch (git push origin my-new-feature)
  6. Create new Pull Request

Owner metadata

GitHub Events

Total
Last Year

Committers metadata

Last synced: about 22 hours ago

Total Commits: 1,685
Total Committers: 83
Avg Commits per committer: 20.301
Development Distribution Score (DDS): 0.484

Commits in past year: 45
Committers in past year: 3
Avg Commits per committer in past year: 15.0
Development Distribution Score (DDS) in past year: 0.244

Name Email Commits
Peter Zotov w****k@w****g 870
Ilya Bylich i****h@g****m 349
Koichi ITO k****o@g****m 93
Alex Dowad a****g@g****m 74
Marc-Andre Lafortune g****b@m****a 51
Markus Schirp m****j@s****m 26
Keiji, Yoshimi w****3@g****m 18
Earlopain 1****n 17
Vladimir Dementyev d****m@g****m 14
Masataka Pocke Kuwabara k****a@p****e 13
Yuji Nakayama n****j@g****m 11
Yorick Peterse y****e@g****m 11
Magnus Holm j****r@g****m 10
Bozhidar Batsov b****r@b****m 8
Bozhidar Batsov b****r@t****m 7
Nelson Elhage n****e@n****m 6
cremno c****o@m****u 6
Olle Jonsson o****n@g****m 5
John Backus j****s@g****m 4
Jon Frisby j****y@m****m 4
Akim Demaille a****e@g****m 3
Benoit Daloze e****p@g****m 3
Denis Defreyne d****e@s****g 3
Josh Cheek j****k@g****m 3
Matijs van Zuijlen m****s@m****t 3
Oleg Zubchenko R****d@g****m 3
Rafa García hi@r****t 3
Ryan Biesemeyer r****n@s****m 3
Jean byroot Boussier j****b@s****m 2
Ian MacLeod i****n@n****t 2
and 53 more...

Committer domains:

Issue and Pull Request metadata

Last synced: 14 days ago

Total issues: 112
Total pull requests: 196
Average time to close issues: 3 months
Average time to close pull requests: 21 days
Total issue authors: 33
Total pull request authors: 29
Average comments per issue: 1.59
Average comments per pull request: 1.44
Merged pull request: 165
Bot issues: 0
Bot pull requests: 0

Past year issues: 4
Past year pull requests: 20
Past year average time to close issues: about 1 month
Past year average time to close pull requests: 12 days
Past year issue authors: 4
Past year pull request authors: 3
Past year average comments per issue: 2.25
Past year average comments per pull request: 2.0
Past year merged pull request: 16
Past year bot issues: 0
Past year bot pull requests: 0

More stats: https://issues.ecosyste.ms/repositories/lookup?url=https://github.com/whitequark/parser

Top Issue Authors

  • iliabylich (60)
  • kddnewton (8)
  • koic (7)
  • Earlopain (5)
  • akimd (2)
  • RazrFalcon (2)
  • hmdne (2)
  • qbantek (1)
  • eregon (1)
  • prikha (1)
  • ko1 (1)
  • bquorning (1)
  • owst (1)
  • daBee (1)
  • faraaz-deepsource (1)

Top Pull Request Authors

  • iliabylich (67)
  • koic (52)
  • Earlopain (34)
  • apiology (4)
  • palkan (4)
  • casperisfine (3)
  • eregon (3)
  • baweaver (2)
  • guillaumewrobel (2)
  • indigolain (2)
  • dgollahon (2)
  • Watson1978 (2)
  • marcandre (2)
  • navidemad (2)
  • alex-tan (1)

Top Issue Labels

  • upstream (58)
  • bug (5)
  • question (1)

Top Pull Request Labels

Package metadata

gem.coop: parser

A Ruby parser written in pure Ruby.

  • Homepage: https://github.com/whitequark/parser
  • Documentation: http://www.rubydoc.info/gems/parser/
  • Licenses: MIT
  • Latest release: 2.2.0 (published about 11 years ago)
  • Last Synced: 2026-03-01T15:03:56.056Z (2 days ago)
  • Versions: 167
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Downloads: 739,946,927 Total
  • Docker Downloads: 3,510,046,988
  • Rankings:
    • Dependent repos count: 0.0%
    • Dependent packages count: 0.0%
    • Average: 0.007%
    • Downloads: 0.022%
  • Maintainers (5)
rubygems.org: parser

A Ruby parser written in pure Ruby.

  • Homepage: https://github.com/whitequark/parser
  • Documentation: http://www.rubydoc.info/gems/parser/
  • Licenses: MIT
  • Latest release: 2.2.0 (published about 11 years ago)
  • Last Synced: 2026-03-02T07:25:04.739Z (1 day ago)
  • Versions: 167
  • Dependent Packages: 304
  • Dependent Repositories: 92,403
  • Downloads: 740,165,375 Total
  • Docker Downloads: 3,510,046,988
  • Rankings:
    • Downloads: 0.03%
    • Docker downloads count: 0.06%
    • Dependent repos count: 0.123%
    • Dependent packages count: 0.146%
    • Average: 0.594%
    • Stargazers count: 1.297%
    • Forks count: 1.907%
  • Maintainers (4)
proxy.golang.org: github.com/whitequark/parser

  • Homepage:
  • Documentation: https://pkg.go.dev/github.com/whitequark/parser#section-documentation
  • Licenses: other
  • Latest release: v2.2.0+incompatible (published about 11 years ago)
  • Last Synced: 2026-03-02T07:25:06.264Z (1 day ago)
  • Versions: 27
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Rankings:
    • Stargazers count: 1.713%
    • Forks count: 1.952%
    • Average: 6.001%
    • Dependent packages count: 9.56%
    • Dependent repos count: 10.779%
debian-11: ruby-whitequark-parser

  • Homepage: https://github.com/whitequark/parser
  • Documentation: https://packages.debian.org/bullseye/ruby-whitequark-parser
  • Licenses:
  • Latest release: 2.7.1.4-2 (published 21 days ago)
  • Last Synced: 2026-02-13T08:26:14.575Z (18 days ago)
  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Rankings:
    • Dependent repos count: 0.0%
    • Dependent packages count: 0.0%
    • Average: 100%
ubuntu-24.04: ruby-whitequark-parser

  • Homepage: https://github.com/whitequark/parser
  • Licenses:
  • Latest release: 3.1.3.0-1 (published 25 days ago)
  • Last Synced: 2026-02-06T16:17:05.763Z (25 days ago)
  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Rankings:
ubuntu-23.04: ruby-whitequark-parser

  • Homepage: https://github.com/whitequark/parser
  • Licenses:
  • Latest release: 3.1.3.0-1 (published 20 days ago)
  • Last Synced: 2026-02-11T06:52:34.958Z (20 days ago)
  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Rankings:
    • Dependent repos count: 0.0%
    • Dependent packages count: 0.0%
    • Average: 100%
ubuntu-23.10: ruby-whitequark-parser

  • Homepage: https://github.com/whitequark/parser
  • Licenses:
  • Latest release: 3.1.3.0-1 (published 18 days ago)
  • Last Synced: 2026-02-13T18:35:44.216Z (18 days ago)
  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Rankings:
    • Dependent repos count: 0.0%
    • Dependent packages count: 0.0%
    • Average: 100%
ubuntu-24.10: ruby-whitequark-parser

  • Homepage: https://github.com/whitequark/parser
  • Licenses:
  • Latest release: 3.3.4.2-1 (published 22 days ago)
  • Last Synced: 2026-02-09T17:32:47.972Z (22 days ago)
  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Rankings:
debian-10: ruby-whitequark-parser

  • Homepage: https://github.com/whitequark/parser
  • Documentation: https://packages.debian.org/buster/ruby-whitequark-parser
  • Licenses:
  • Latest release: 2.4.0.2-1 (published 20 days ago)
  • Last Synced: 2026-02-13T04:27:00.334Z (19 days ago)
  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Rankings:
    • Dependent repos count: 0.0%
    • Dependent packages count: 0.0%
    • Average: 100%
guix: ruby-parser

Ruby parser written in pure Ruby

  • Homepage: https://github.com/whitequark/parser
  • Documentation: https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/ruby-xyz.scm#n6496
  • Licenses: expat
  • Latest release: 3.3.7.0 (published about 23 hours ago)
  • Last Synced: 2026-03-02T18:56:15.128Z (about 23 hours ago)
  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Rankings:
    • Dependent repos count: 0.0%
    • Dependent packages count: 0.0%
    • Average: 100%
ubuntu-20.04: ruby-whitequark-parser

  • Homepage: https://github.com/whitequark/parser
  • Licenses:
  • Latest release: 2.7.0.2-1 (published 18 days ago)
  • Last Synced: 2026-02-13T07:25:35.058Z (18 days ago)
  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Rankings:
    • Dependent repos count: 0.0%
    • Dependent packages count: 0.0%
    • Average: 100%
ubuntu-22.04: ruby-whitequark-parser

  • Homepage: https://github.com/whitequark/parser
  • Licenses:
  • Latest release: 3.0.2.0-3 (published 18 days ago)
  • Last Synced: 2026-02-13T13:28:38.917Z (18 days ago)
  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Rankings:
    • Dependent repos count: 0.0%
    • Dependent packages count: 0.0%
    • Average: 100%
debian-12: ruby-whitequark-parser

  • Homepage: https://github.com/whitequark/parser
  • Documentation: https://packages.debian.org/bookworm/ruby-whitequark-parser
  • Licenses:
  • Latest release: 3.1.3.0-1 (published 19 days ago)
  • Last Synced: 2026-02-12T23:44:14.116Z (19 days ago)
  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Rankings:
    • Dependent repos count: 0.0%
    • Dependent packages count: 0.0%
    • Average: 100%
debian-13: ruby-whitequark-parser

  • Homepage: https://github.com/whitequark/parser
  • Documentation: https://packages.debian.org/trixie/ruby-whitequark-parser
  • Licenses:
  • Latest release: 3.3.4.2-2 (published 19 days ago)
  • Last Synced: 2026-02-13T13:20:51.208Z (18 days ago)
  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Rankings:
    • Dependent repos count: 0.0%
    • Dependent packages count: 0.0%
    • Average: 100%

Dependencies

.github/workflows/test.yml actions
  • actions/checkout v3 composite
  • ruby/setup-ruby v1 composite
Gemfile rubygems
  • ast >= 1.1, < 3.0
parser.gemspec rubygems
  • bundler >= 1.15, < 3.0.0 development
  • cliver ~> 0.3.2 development
  • gauntlet >= 0 development
  • kramdown >= 0 development
  • minitest ~> 5.10 development
  • racc = 1.6.1 development
  • rake ~> 13.0.1 development
  • simplecov ~> 0.15.1 development
  • yard >= 0 development
  • ast ~> 2.4.1

Score: 34.70577690103235