[wasm][asm.js] Asm.js -> wasm custom parser.

src/asmjs/asm-parser.h

+// Copyright 2017 the V8 project authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#ifndef V8_ASMJS_ASM_PARSER_H_
+#define V8_ASMJS_ASM_PARSER_H_
+
+#include <list>
+#include <string>
+#include <vector>
+
+#include "src/asmjs/asm-scanner.h"
+#include "src/asmjs/asm-typer.h"
+#include "src/asmjs/asm-types.h"
+#include "src/wasm/signature-map.h"
+#include "src/wasm/wasm-module-builder.h"
+
+namespace v8 {
+namespace internal {
+namespace wasm {
+
+class AsmJsParser {

[Michael Starzinger, 2017/03/22 14:55:51]

nit: Can we add a high-level description of what the intent of this class is. I
was thinking about mentioning that it performs parsing, asm.js validation and
translation to WASM in (almost) a single pass.

[bradn, 2017/03/24 03:58:19]

Done.

+ public:
+  explicit AsmJsParser(Isolate* isolate, Zone* zone, Handle<Script> script,
+                       int start, int end);
+  bool Run();
+  const char* failure_message() const { return failure_message_.c_str(); }
+  int failure_location() const { return failure_location_; }
+  WasmModuleBuilder* builder() { return builder_; }
+  const AsmTyper::StdlibSet* stdlib_uses() const { return &stdlib_uses_; }
+
+ private:
+  Zone* zone_;

[vogelheim, 2017/03/17 17:27:06]

The style guide asks to "generally prefer grouping similar kinds of declarations
together" (i.e., methods, then member variables, etc.) It does allow for
exceptions ("generally"), and here they're mixed quite wildly. Not sure if this
helps readability... :-|

[Michael Starzinger, 2017/03/22 14:55:51]

+1. Lets group similar declaration together, all fields at the bottom, all
simple accessors together.

[bradn, 2017/03/24 03:58:19]

Done.

[bradn, 2017/03/24 03:58:19]

Done.

+  Zone* zone() { return zone_; }
+
+  AsmJsScanner scanner_;
+  WasmModuleBuilder* builder_;
+  WasmFunctionBuilder* current_function_builder_;
+  AsmType* return_type_;
+  std::uintptr_t stack_limit_;
+  AsmTyper::StdlibSet stdlib_uses_;
+
+  // Error Handling.
+  bool failed_;
+  std::string failure_message_;
+  int failure_location_;
+
+  // clang-format off
+  enum class VarKind {
+    kUnused,
+    kLocal,
+    kGlobal,
+    kSpecial,
+    kFunction,
+    kTable,
+    kImportedFunction,
+#define V(_unused0, Name, _unused1, _unused2) kMath##Name,
+    STDLIB_MATH_FUNCTION_LIST(V)
+#undef V
+#define V(Name) kMath##Name,
+    STDLIB_MATH_VALUE_LIST(V)
+#undef V
+  };
+  // clang-format on
+
+  struct FunctionImportInfo {
+    std::string name;
+    SignatureMap cache;
+    std::vector<uint32_t> cache_index;
+  };
+  std::list<FunctionImportInfo> function_import_info_;
+
+  struct VarInfo {
+    AsmType* type;
+    uint32_t index;

[vogelheim, 2017/03/17 17:27:07]

Not sure if worthwhile:

On 64b platforms this member ordering is somewhat wasteful due to alignment
rules. I'd expect there to be fair amount of VarInfo's in one parse, so it might
be worthwile to reorder this to avoid 'holes' in the struct.

[bradn, 2017/03/24 03:58:19]

Done.

+    uint64_t mask;
+    VarKind kind;
+    WasmFunctionBuilder* function_builder;
+    bool mut;
+    FunctionImportInfo* import;
+
+    VarInfo();
+    void DeclareGlobalImport(AsmType* type, uint32_t index);
+    void DeclareStdlibFunc(VarKind kind, AsmType* type);
+  };
+  std::vector<VarInfo> global_var_info_;

[marja, 2017/03/17 13:49:04]

std containers vs. Zone containers... expanding vectors in Zone are bad for
memory, so, it would need some trickery to store stuff in the Zone. Otoh getting
rid of the Zone memory is much faster. What's your take on this trade off in the
asm js parser?

[bradn, 2017/03/24 03:58:19]

I suspect switching to Zone allocation might be a win for the locals, but might
also require tuning to figure out a good start size.
For the globals, they stick around for the whole time, so probably is moot for
them.

+  std::vector<VarInfo> local_var_info_;
+  VarInfo* GetVarInfo(AsmJsScanner::token_t token);
+  uint32_t VarIndex(VarInfo* info);
+  void DeclareGlobal(VarInfo* info, bool mut, AsmType* type, ValueType vtype,

[vogelheim, 2017/03/17 17:27:06]

mut?

[bradn, 2017/03/24 03:58:19]

mutable, but clashes with the c++ keyword.
changed to mutable_variable.

+                     const WasmInitExpr& init = WasmInitExpr());
+
+  // Manage temporaries.
+  int function_temp_locals_offset_;
+  int function_temp_locals_used_;
+  int32_t TempVariable(int i);
+
+  // Global imports.
+  // NOTE: Holds the strings referenced in wasm-module-builder for imports.
+  struct GlobalImport {
+    std::string import_name;
+    uint32_t import_index;
+    uint32_t global_index;
+    bool needs_init;
+  };
+  std::list<GlobalImport> global_imports_;

[Michael Starzinger, 2017/03/22 14:55:51]

nit: Could we use ZoneLinkedList here?

[bradn, 2017/03/24 03:58:19]

Done.

+  void AddGlobalImport(std::string name, AsmType* type, ValueType vtype,
+                       bool mut, VarInfo* info);
+
+  // Module Related.
+  AsmJsScanner::token_t stdlib_name_;
+  AsmJsScanner::token_t foreign_name_;
+  AsmJsScanner::token_t heap_name_;
+
+  bool Peek(AsmJsScanner::token_t token) { return scanner_.Token() == token; }
+  AsmJsScanner::token_t Consume() {
+    AsmJsScanner::token_t ret = scanner_.Token();
+    scanner_.Next();
+    return ret;
+  }
+  void SkipSemicolon();
+
+  // Block stack.
+  enum class BlockKind { kRegular, kLoop, kOther };
+  struct BlockInfo {
+    BlockKind kind;
+    AsmJsScanner::token_t label;
+  };
+  std::vector<BlockInfo> block_stack_;

[Michael Starzinger, 2017/03/22 14:55:51]

This really seems to be used as a stack, would it make sense to use a
{ZoneStack} instead? WDYT?

[bradn, 2017/03/24 03:58:19]

It unfortunately walks the stack to scan for labels (without unwinding it).

+  void Begin(AsmJsScanner::token_t label = 0);
+  void Loop(AsmJsScanner::token_t label = 0);
+  void End();
+  void BareBegin(BlockKind kind = BlockKind::kOther,

[vogelheim, 2017/03/17 17:27:06]

What's a Bare (as in: BareBegin / BareEnd)? I take it it's a syntactic
construct, but I can't figure out which one.

[bradn, 2017/03/24 03:58:19]

Not happy with these names myself.
The Bare* ones just add block_stack_ layers, Begin/Loop/End actually emit wasm
blocks/loops.

Part of the naming issue is too much "block": the wasm opcode block, the general
category that loop + block belong to, the javascript block and the method to
parse it below.

Added a comment, but welcome suggestions.

+                 AsmJsScanner::token_t label = 0);
+  void BareEnd();
+  int FindLabelDepth(AsmJsScanner::token_t label, bool loop);
+
+  // Track if parsing a heap assignment.
+  bool inside_heap_assignment_;
+  int heap_assignment_location_;
+  AsmType* heap_access_type_;
+
+  AsmType* call_coercion_;

[marja, 2017/03/21 13:31:44]

Pls add a comment about what this is?

[bradn, 2017/03/24 03:58:19]

Done.

+
+  AsmJsScanner::token_t pending_label_;
+
+  // Types used for stdlib function and their set up.
+  AsmType* stdlib_dq2d_;
+  AsmType* stdlib_dqdq2d_;
+  AsmType* stdlib_fq2f_;
+  AsmType* stdlib_i2s_;
+  AsmType* stdlib_ii2s_;
+  AsmType* stdlib_minmax_;
+  AsmType* stdlib_abs_;
+  AsmType* stdlib_ceil_like_;
+  AsmType* stdlib_fround_;
+  void InitializeStdlibTypes();
+
+  FunctionSig* ConvertSignature(AsmType* return_type,
+                                const std::vector<AsmType*>& params);
+
+  // 6.1 ValidateModule
+  void ValidateModule();
+  void ValidateModuleParameters();
+  void ValidateModuleVars();
+  void ValidateModuleVar(bool mut);
+  void ValidateModuleVarStdlib(VarInfo* info);
+
+  void ValidateExport();                 // 6.2 ValidateExport
+  void ValidateFunctionTable();          // 6.3 ValidateFunctionTable
+  void ValidateFunction();               // 6.4 ValidateFunction
+  void ValidateStatement();              // ValidateStatement
+  void Block();                          // 6.5.1 Block

[marja, 2017/03/17 13:49:04]

Naming nit: would it make sense to name these ParseBlock etc. for consistency w/
Parser?

[bradn, 2017/03/24 03:58:19]

Unsure, had named them this way to mirror the asm.js spec.

+  void ExpressionStatement();            // 6.5.2 ExpressionStatement
+  void EmptyStatement();                 // 6.5.3 EmptyStatement
+  void IfStatement();                    // 6.5.4 IfStatement
+  void ReturnStatement();                // 6.5.5 ReturnStatement
+  bool IterationStatement();             // 6.5.6 IterationStatement
+  void WhileStatement();                 // 6.5.6 IterationStatement - while
+  void DoStatement();                    // 6.5.6 IterationStatement - do
+  void ForStatement();                   // 6.5.6 IterationStatement - for
+  void BreakStatement();                 // 6.5.7 BreakStatement
+  void ContinueStatement();              // 6.5.8 ContinueStatement
+  void LabelledStatement();              // 6.5.9 LabelledStatement
+  void SwitchStatement();                // 6.5.10 SwitchStatement
+  void ValidateCase();                   // 6.6. ValidateCase
+  void ValidateDefault();                // 6.7 ValidateDefault
+  AsmType* ValidateExpression();         // 6.8 ValidateExpression
+  AsmType* Expression(AsmType* expect);  // 6.8.1 Expression
+  AsmType* NumericLiteral();             // 6.8.2 NumericLiteral
+  AsmType* Identifier();                 // 6.8.3 Identifier
+  AsmType* CallExpression();             // 6.8.4 CallExpression
+  AsmType* MemberExpression();           // 6.8.5 MemberExpression
+  AsmType* AssignmentExpression();       // 6.8.6 AssignmentExpression
+  AsmType* UnaryExpression();            // 6.8.7 UnaryExpression
+  AsmType* MultiplicativeExpression();   // 6.8.8 MultaplicativeExpression
+  AsmType* AdditiveExpression();         // 6.8.9 AdditiveExpression
+  AsmType* ShiftExpression();            // 6.8.10 ShiftExpression
+  AsmType* RelationalExpression();       // 6.8.11 RelationalExpression
+  AsmType* EqualityExpression();         // 6.8.12 EqualityExpression
+  AsmType* BitwiseANDExpression();       // 6.8.13 BitwiseANDExpression
+  AsmType* BitwiseXORExpression();       // 6.8.14 BitwiseXORExpression
+  AsmType* BitwiseORExpression();        // 6.8.15 BitwiseORExpression
+  AsmType* ConditionalExpression();      // 6.8.16 ConditionalExpression
+  AsmType* ParenthesizedExpression();    // 6.8.17 ParenthesiedExpression
+  AsmType* ValidateCall();               // 6.9 ValidateCall
+  bool PeekCall();                       // 6.9 ValidateCall - helper
+  void ValidateHeapAccess();             // 6.10 ValidateHeapAccess
+  void ValidateFloatCoercion();          // 6.11 ValidateFloatCoercion
+
+  void GatherCases(std::vector<int32_t>* cases);
+};
+
+}  // namespace wasm
+}  // namespace internal
+}  // namespace v8
+#endif