class Prism::Translation::Parser

此类是用于将 prism 语法树转换为 whitequark/parser gem 的语法树的入口点。它继承自 parser gem 的基础解析器，并重写了 parse* 方法，以便使用 prism 进行解析然后进行翻译。

请注意，此版本的解析器始终使用 Prism 支持的最新 Ruby 语法版本进行解析。如果您想要特定的版本支持，请使用版本特定的子类，例如 Prism::Translation::Parser34。如果您想使用与当前运行的 Ruby 版本相同的 Ruby 语法版本进行解析，请使用 Prism::Translation::ParserCurrent。

Public Class Methods

new (builder = Prism::Translation::Parser::Builder.new, parser: Prism)

Source

# File lib/prism/translation/parser.rb, line 74
      def initialize(builder = Prism::Translation::Parser::Builder.new, parser: Prism)
        if !builder.is_a?(Prism::Translation::Parser::Builder)
          warn(<<~MSG, uplevel: 1, category: :deprecated)
            [deprecation]: The builder passed to `Prism::Translation::Parser.new` is not a \
            `Prism::Translation::Parser::Builder` subclass. This will raise in the next major version.
          MSG
        end
        @parser = parser

        super(builder)
      end

builder 参数默认用于使用我们自定义的 builder 类来创建解析器。

通过使用 :parser 关键字参数，您可以使用任何解析器以与 Parser gem 兼容的方式进行翻译。

例如，在 RuboCop for Ruby LSP 中，可以使用以下方法通过重用预先解析的 Prism::ParseLexResult 来提高性能。

class PrismPreparsed def initialize(prism_result) @prism_result = prism_result end

def parse_lex(source, **options)
  @prism_result
end

end

prism_preparsed = PrismPreparsed.new(prism_result)

Prism::Translation::Ruby34.new(builder, parser: prism_preparsed)

在传递给 :parser 关键字参数的对象中，应根据需要实现 parse 和 parse_lex 方法。

调用超类方法

Public Instance Methods

default_encoding ()

Source

# File lib/prism/translation/parser.rb, line 91
def default_encoding
  Encoding::UTF_8
end

Ruby 文件的默认编码是 UTF-8。

parse (source_buffer)

Source

# File lib/prism/translation/parser.rb, line 99
def parse(source_buffer)
  @source_buffer = source_buffer
  source = source_buffer.source

  offset_cache = build_offset_cache(source)
  result = unwrap(@parser.parse(source, **prism_options), offset_cache)

  build_ast(result.value, offset_cache)
ensure
  @source_buffer = nil
end

解析源缓冲区并返回 AST。

parse_with_comments (source_buffer)

Source

# File lib/prism/translation/parser.rb, line 112
def parse_with_comments(source_buffer)
  @source_buffer = source_buffer
  source = source_buffer.source

  offset_cache = build_offset_cache(source)
  result = unwrap(@parser.parse(source, **prism_options), offset_cache)

  [
    build_ast(result.value, offset_cache),
    build_comments(result.comments, offset_cache)
  ]
ensure
  @source_buffer = nil
end

解析源缓冲区并返回 AST 和源代码注释。

tokenize (source_buffer, recover = false)

Source

# File lib/prism/translation/parser.rb, line 129
def tokenize(source_buffer, recover = false)
  @source_buffer = source_buffer
  source = source_buffer.source

  offset_cache = build_offset_cache(source)
  result =
    begin
      unwrap(@parser.parse_lex(source, **prism_options), offset_cache)
    rescue ::Parser::SyntaxError
      raise if !recover
    end

  program, tokens = result.value
  ast = build_ast(program, offset_cache) if result.success?

  [
    ast,
    build_comments(result.comments, offset_cache),
    build_tokens(tokens, offset_cache)
  ]
ensure
  @source_buffer = nil
end

解析源缓冲区并返回 AST、源代码注释以及 lexer 发出的 token。

try_declare_numparam (node)

Source

# File lib/prism/translation/parser.rb, line 155
def try_declare_numparam(node)
  node.children[0].match?(/\A_[1-9]\z/)
end

由于 prism 会为我们解析 num params，因此我们无需在此处支持此类逻辑。

私有实例方法

build_ast (program, offset_cache)

Source

# File lib/prism/translation/parser.rb, line 313
def build_ast(program, offset_cache)
  program.accept(Compiler.new(self, offset_cache))
end

从 prism AST 构建 parser gem AST。

build_comments (comments, offset_cache)

Source

# File lib/prism/translation/parser.rb, line 318
def build_comments(comments, offset_cache)
  comments.map do |comment|
    ::Parser::Source::Comment.new(build_range(comment.location, offset_cache))
  end
end

从 prism 注释构建 parser gem 注释。

build_offset_cache (source)

Source

# File lib/prism/translation/parser.rb, line 296
def build_offset_cache(source)
  if source.bytesize == source.length
    -> (offset) { offset }
  else
    offset_cache = []
    offset = 0

    source.each_char do |char|
      char.bytesize.times { offset_cache << offset }
      offset += 1
    end

    offset_cache << offset
  end
end

Prism 以字节为单位处理偏移量，而 parser gem 以字符为单位处理偏移量。我们需要处理此转换才能构建 parser gem AST。

如果源的字节大小与长度相同，则我们可以直接使用偏移量。否则，我们将构建一个数组，其中索引是字节偏移量，值是字符偏移量。

build_range (location, offset_cache)

Source

# File lib/prism/translation/parser.rb, line 330
def build_range(location, offset_cache)
  ::Parser::Source::Range.new(
    source_buffer,
    offset_cache[location.start_offset],
    offset_cache[location.end_offset]
  )
end

从 prism 位置构建一个范围。

build_tokens (tokens, offset_cache)

Source

# File lib/prism/translation/parser.rb, line 325
def build_tokens(tokens, offset_cache)
  Lexer.new(source_buffer, tokens, offset_cache).to_a
end

从 prism token 构建 parser gem token。

convert_for_prism (version)

Source

# File lib/prism/translation/parser.rb, line 353
def convert_for_prism(version)
  case version
  when 33
    "3.3.1"
  when 34
    "3.4.0"
  when 35, 40
    "4.0.0"
  when 41
    "4.1.0"
  else
    "latest"
  end
end

将 Parser 处理的版本格式转换为 Prism 处理的格式。

error_diagnostic (error, offset_cache)

Source

# File lib/prism/translation/parser.rb, line 174
def error_diagnostic(error, offset_cache)
  location = error.location
  diagnostic_location = build_range(location, offset_cache)

  case error.type
  when :argument_block_multi
    Diagnostic.new(:error, :block_and_blockarg, {}, diagnostic_location, [])
  when :argument_formal_constant
    Diagnostic.new(:error, :argument_const, {}, diagnostic_location, [])
  when :argument_formal_class
    Diagnostic.new(:error, :argument_cvar, {}, diagnostic_location, [])
  when :argument_formal_global
    Diagnostic.new(:error, :argument_gvar, {}, diagnostic_location, [])
  when :argument_formal_ivar
    Diagnostic.new(:error, :argument_ivar, {}, diagnostic_location, [])
  when :argument_no_forwarding_amp
    Diagnostic.new(:error, :no_anonymous_blockarg, {}, diagnostic_location, [])
  when :argument_no_forwarding_star
    Diagnostic.new(:error, :no_anonymous_restarg, {}, diagnostic_location, [])
  when :argument_no_forwarding_star_star
    Diagnostic.new(:error, :no_anonymous_kwrestarg, {}, diagnostic_location, [])
  when :begin_lonely_else
    location = location.copy(length: 4)
    diagnostic_location = build_range(location, offset_cache)
    Diagnostic.new(:error, :useless_else, {}, diagnostic_location, [])
  when :class_name, :module_name
    Diagnostic.new(:error, :module_name_const, {}, diagnostic_location, [])
  when :class_in_method
    Diagnostic.new(:error, :class_in_def, {}, diagnostic_location, [])
  when :def_endless_setter
    Diagnostic.new(:error, :endless_setter, {}, diagnostic_location, [])
  when :embdoc_term
    Diagnostic.new(:error, :embedded_document, {}, diagnostic_location, [])
  when :incomplete_variable_class, :incomplete_variable_class_3_3
    location = location.copy(length: location.length + 1)
    diagnostic_location = build_range(location, offset_cache)

    Diagnostic.new(:error, :cvar_name, { name: location.slice }, diagnostic_location, [])
  when :incomplete_variable_instance, :incomplete_variable_instance_3_3
    location = location.copy(length: location.length + 1)
    diagnostic_location = build_range(location, offset_cache)

    Diagnostic.new(:error, :ivar_name, { name: location.slice }, diagnostic_location, [])
  when :invalid_variable_global, :invalid_variable_global_3_3
    Diagnostic.new(:error, :gvar_name, { name: location.slice }, diagnostic_location, [])
  when :module_in_method
    Diagnostic.new(:error, :module_in_def, {}, diagnostic_location, [])
  when :numbered_parameter_ordinary
    Diagnostic.new(:error, :ordinary_param_defined, {}, diagnostic_location, [])
  when :numbered_parameter_outer_scope
    Diagnostic.new(:error, :numparam_used_in_outer_scope, {}, diagnostic_location, [])
  when :parameter_circular
    Diagnostic.new(:error, :circular_argument_reference, { var_name: location.slice }, diagnostic_location, [])
  when :parameter_name_repeat
    Diagnostic.new(:error, :duplicate_argument, {}, diagnostic_location, [])
  when :parameter_numbered_reserved
    Diagnostic.new(:error, :reserved_for_numparam, { name: location.slice }, diagnostic_location, [])
  when :regexp_unknown_options
    Diagnostic.new(:error, :regexp_options, { options: location.slice[1..] }, diagnostic_location, [])
  when :singleton_for_literals
    Diagnostic.new(:error, :singleton_literal, {}, diagnostic_location, [])
  when :string_literal_eof
    Diagnostic.new(:error, :string_eof, {}, diagnostic_location, [])
  when :unexpected_token_ignore
    Diagnostic.new(:error, :unexpected_token, { token: location.slice }, diagnostic_location, [])
  when :write_target_in_method
    Diagnostic.new(:error, :dynamic_const, {}, diagnostic_location, [])
  else
    PrismDiagnostic.new(error.message, :error, error.type, diagnostic_location)
  end
end

从给定的 prism 解析错误构建一个诊断。

prism_options ()

Source

# File lib/prism/translation/parser.rb, line 339
def prism_options
  options = {
    filepath: @source_buffer.name,
    version: convert_for_prism(version),
    partial_script: true,
  }
  # The parser gem always encodes to UTF-8, unless it is binary.
  # https://github.com/whitequark/parser/blob/v3.3.6.0/lib/parser/source/buffer.rb#L80-L107
  options[:encoding] = false if @source_buffer.source.encoding != Encoding::BINARY

  options
end

关于 prism 应如何解析/词法化源的选项。

unwrap (result, offset_cache)

Source

# File lib/prism/translation/parser.rb, line 274
def unwrap(result, offset_cache)
  result.errors.each do |error|
    next unless valid_error?(error)
    diagnostics.process(error_diagnostic(error, offset_cache))
  end

  result.warnings.each do |warning|
    next unless valid_warning?(warning)
    diagnostic = warning_diagnostic(warning, offset_cache)
    diagnostics.process(diagnostic) if diagnostic
  end

  result
end

如果在解析过程中生成了错误，则会引发相应的语法错误。否则返回结果。

valid_error? (error)

Source

# File lib/prism/translation/parser.rb, line 163
def valid_error?(error)
  true
end

这是一个钩子，允许使用者在不希望某些错误阻止创建语法树时禁用它们。

valid_warning? (warning)

Source

# File lib/prism/translation/parser.rb, line 169
def valid_warning?(warning)
  true
end

这是一个钩子，允许使用者在不希望某些警告阻止创建语法树时禁用它们。

warning_diagnostic (warning, offset_cache)

Source

# File lib/prism/translation/parser.rb, line 247
def warning_diagnostic(warning, offset_cache)
  diagnostic_location = build_range(warning.location, offset_cache)

  case warning.type
  when :ambiguous_first_argument_plus
    Diagnostic.new(:warning, :ambiguous_prefix, { prefix: "+" }, diagnostic_location, [])
  when :ambiguous_first_argument_minus
    Diagnostic.new(:warning, :ambiguous_prefix, { prefix: "-" }, diagnostic_location, [])
  when :ambiguous_prefix_ampersand
    Diagnostic.new(:warning, :ambiguous_prefix, { prefix: "&" }, diagnostic_location, [])
  when :ambiguous_prefix_star
    Diagnostic.new(:warning, :ambiguous_prefix, { prefix: "*" }, diagnostic_location, [])
  when :ambiguous_prefix_star_star
    Diagnostic.new(:warning, :ambiguous_prefix, { prefix: "**" }, diagnostic_location, [])
  when :ambiguous_slash
    Diagnostic.new(:warning, :ambiguous_regexp, {}, diagnostic_location, [])
  when :dot_dot_dot_eol
    Diagnostic.new(:warning, :triple_dot_at_eol, {}, diagnostic_location, [])
  when :duplicated_hash_key
    # skip, parser does this on its own
  else
    PrismDiagnostic.new(warning.message, :warning, warning.type, diagnostic_location)
  end
end

从给定的 prism 解析警告构建一个诊断。