feat: suggest corrections for unbound identifiers/fields based on dis…

…tance metric (#4720) Add flag --ai-errors to tailor error reporting for human (default) or artificial users. For humans, instead of reporting that a value or type identifier is not available in references or projections, we now suggest likely candidates based on: * whether the identifier is a prefix of the candidate identifiers. * whether the identifier is some [edit distance](https://en.wikipedia.org/wiki/Levenshtein_distance) away from the candidate identifiers. and report identifiers in order of edit distance. When the object/actor/module projected from has few bindings (<16), we display them with their types, to avoid producing huge error messages. With --ai-errors, we don't give hints but display all available bindings with their types (Thanks to Claude.ai for coughing up an implementation.)
dfinity · Oct 4, 2024 · 15a4ac2 · 15a4ac2
1 parent 632f5f2
commit 15a4ac2
Show file tree

Hide file tree

Showing 49 changed files with 811 additions and 31 deletions.
diff --git a/Changelog.md b/Changelog.md
@@ -1,5 +1,11 @@
 # Motoko compiler changelog
 
+* motoko (`moc`)
+
+  * Improved error messages for unbound identifiers and fields that avoid reporting large types and use an edit-distance based metric to suggest alternatives (#4720).
+
+  * Flag `--ai-errors` to tailor error messages to AI clients (#4720).
+
 ## 0.13.0 (2024-09-17)
 
 * motoko (`moc`)
@@ -13,7 +19,7 @@
     * The Wasm main memory (heap) is retained on upgrade with new program versions directly picking up this state.
     * The Wasm main memory has been extended to 64-bit to scale as large as stable memory in the future.
     * The runtime system checks that data changes of new program versions are compatible with the old state.
-    
+
     Implications:
     * Upgrades become extremely fast, only depending on the number of types, not on the number of heap objects.
     * Upgrades will no longer hit the IC instruction limit, even for maximum heap usage.

diff --git a/doc/md/reference/compiler-ref.md b/doc/md/reference/compiler-ref.md
@@ -26,6 +26,7 @@ You can use the following options with the `moc` command.
 
 | Option                                    | Description                                                                                                                                           |
 |-------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `--ai-errors`                             | Emit AI tailored error messages.                                                                                                                      |
 | `--actor-idl <idl-path>`                  | Specifies a path to actor IDL (Candid) files.                                                                                                         |
 | `--actor-alias <alias> <principal>`       | Specifies an actor import alias.                                                                                                                      |
 | `--args <file>`                           | Read additional newline separated command line arguments from `<file>`.                                                                               |
@@ -39,12 +40,12 @@ You can use the following options with the `moc` command.
 | `--error-detail <n>`                      | Set level of error message detail for syntax errors, n in \[0..3\] (default 2).                                                                       |
 | `--experimental-stable-memory <n>`        | Select support for the deprecated `ExperimentalStableMemory.mo` library (n < 0: error, n = 0: warn, n > 0: allow) (default 0).                        |
 | `-fno-shared-code`                        | Do not share low-level utility code: larger code size but decreased cycle consumption (default).                                                      |
-| `--generational-gc`                       | Use generational GC (not supported with enhanced orthogonal persistence).                                                                              |
+| `--generational-gc`                       | Use generational GC (not supported with enhanced orthogonal persistence).                                                                             |
 | `-fshared-code`                           | Do share low-level utility code: smaller code size but increased cycle consumption.                                                                   |
 | `-help`,`--help`                          | Displays usage information.                                                                                                                           |
 | `--hide-warnings`                         | Hides compiler warnings.                                                                                                                              |
 | `-Werror`                                 | Treat warnings as errors.                                                                                                                             |
-| `--incremental-gc`                        | Use incremental GC (default of enhanced orthogonal persistence, also available for classical persistence).                                             |
+| `--incremental-gc`                        | Use incremental GC (default of enhanced orthogonal persistence, also available for classical persistence).                                            |
 | `--idl`                                   | Compile binary and emit Candid IDL specification to `.did` file.                                                                                      |
 | `-i`                                      | Runs the compiler in an interactive read–eval–print loop (REPL) shell so you can evaluate program execution (implies -r).                             |
 | `--map`                                   | Outputs a JavaScript source map.                                                                                                                      |

diff --git a/src/exes/moc.ml b/src/exes/moc.ml
@@ -37,6 +37,7 @@ let valid_metadata_names =
      "motoko:compiler"]
 
 let argspec = [
+  "--ai-errors", Arg.Set Flags.ai_errors, " emit AI tailored errors";
   "-c", Arg.Unit (set_mode Compile), " compile programs to WebAssembly";
   "-g", Arg.Set Flags.debug_info, " generate source-level debug information";
   "-r", Arg.Unit (set_mode Run), " interpret programs";

diff --git a/src/lib/lib.ml b/src/lib/lib.ml
@@ -236,6 +236,40 @@ struct
       | c -> Buffer.add_char buf c
     done;
     Buffer.contents buf
+
+  (* Courtesy of Claude.ai *)
+  let levenshtein_distance s t =
+    let m = String.length s
+    and n = String.length t in
+
+    (* Ensure s is the shorter string for optimization *)
+    let (s, t, m, n) = if m > n then (t, s, n, m) else (s, t, m, n) in
+
+    (* Initialize the previous row *)
+    let previous_row = Array.init (m + 1) (fun i -> i) in
+
+    (* Compute the distance *)
+    for i = 1 to n do
+      let current_row = Array.make (m + 1) 0 in
+      current_row.(0) <- i;
+
+      for j = 1 to m do
+        let cost = if s.[j-1] = t.[i-1] then 0 else 1 in
+        current_row.(j) <- min
+          (min
+            (previous_row.(j) + 1)     (* Deletion *)
+            (current_row.(j-1) + 1)    (* Insertion *)
+          )
+          (previous_row.(j-1) + cost)  (* Substitution *)
+      done;
+
+      (* Swap rows *)
+      Array.blit current_row 0 previous_row 0 (m + 1)
+    done;
+
+    (* Return the distance *)
+    previous_row.(m)
+
 end
 
 module Utf8 =

diff --git a/src/lib/lib.mli b/src/lib/lib.mli
@@ -160,6 +160,7 @@ sig
   val chop_prefix : string -> string -> string option
   val chop_suffix : string -> string -> string option
   val lightweight_escaped : string -> string
+  val levenshtein_distance : string -> string -> int
 end
 
 module CRC :

diff --git a/src/mo_config/flags.ml b/src/mo_config/flags.ml
@@ -11,6 +11,7 @@ type instruction_limits = {
   update_call: int;
 }
 
+let ai_errors = ref false
 let trace = ref false
 let verbose = ref false
 let print_warnings = ref true

diff --git a/src/mo_frontend/typing.ml b/src/mo_frontend/typing.ml
@@ -114,6 +114,32 @@ let kind_of_field_pattern pf = match pf.it with
   | { id; pat = { it = VarP pat_id; _ } } when id = pat_id -> Scope.FieldReference
   | _ -> Scope.Declaration
 
+(* Suggestions *)
+
+let suggest desc id ids =
+  if !Flags.ai_errors then
+    Printf.sprintf
+      "\nThe %s %s is not available. Try something else?"
+      desc
+      id
+  else
+  let suggestions =
+    let limit = Lib.Int.log2 (String.length id) in
+    let distance = Lib.String.levenshtein_distance id in
+    let weighted_ids = List.filter_map (fun id0 ->
+      let d = distance id0 in
+      if Lib.String.starts_with id id0 || d <= limit then
+        Some (d, id0)
+      else None) ids in
+    List.sort compare weighted_ids |> List.map snd
+  in
+  if suggestions = [] then ""
+  else
+    let rest, last = Lib.List.split_last suggestions in
+    Printf.sprintf "\nDid you mean %s %s?"
+      desc
+      ((if rest <> [] then (String.concat ", " rest) ^ " or " else "") ^ last)
+
 (* Error bookkeeping *)
 
 exception Recover
@@ -128,6 +154,54 @@ let display_typ = Lib.Format.display T.pp_typ
 
 let display_typ_expand = Lib.Format.display T.pp_typ_expand
 
+let display_obj fmt (s, fs) =
+  if !Flags.ai_errors || (List.length fs) < 16 then
+    Format.fprintf fmt "type:%a" display_typ (T.Obj(s, fs))
+  else
+    Format.fprintf fmt "%s." (String.trim(T.string_of_obj_sort s))
+
+let display_vals fmt vals =
+  if !Flags.ai_errors then
+    let tfs = T.Env.fold (fun x (t, _, _, _) acc ->
+      if x = "Prim" || (String.length x >= 0 && x.[0] = '@')
+      then acc
+      else T.{lab = x; src = {depr = None; region = Source.no_region }; typ = t}::acc)
+      vals []
+    in
+    let ty = T.Obj(T.Object, List.rev tfs) in
+    Format.fprintf fmt " in environment:%a" display_typ ty
+  else
+    Format.fprintf fmt ""
+
+let display_labs fmt labs =
+  if !Flags.ai_errors then
+    let tfs = T.Env.fold (fun x t acc ->
+      T.{lab = x; src = {depr = None; region = Source.no_region }; typ = t}::acc)
+      labs []
+    in
+    let ty = T.Obj(T.Object, List.rev tfs) in
+    Format.fprintf fmt " in label environment:%a" display_typ ty
+  else
+    Format.fprintf fmt ""
+
+let display_typs fmt typs =
+  if !Flags.ai_errors then
+    let tfs = T.Env.fold (fun x c acc ->
+      if (String.length x >= 0 && (x.[0] = '@' || x.[0] = '$')) ||
+        T.(match Cons.kind c with
+          | Def ([], Prim _)
+          | Def ([], Any)
+          | Def ([], Non) -> string_of_con c = x
+          | _ -> false)
+      then acc
+      else T.{lab = x; src = {depr = None; region = Source.no_region }; typ = T.Typ c}::acc)
+      typs []
+    in
+    let ty = T.Obj(T.Object, List.rev tfs) in
+    Format.fprintf fmt " in type environment:%a" display_typ ty
+  else
+    Format.fprintf fmt ""
+
 let type_error at code text : Diag.message =
   Diag.error_message at code "type" text
 
@@ -398,8 +472,11 @@ and check_obj_path' env path : T.typ =
        error env id.at "M0024" "cannot infer type of forward variable reference %s" id.it
      | Some (t, _, _, Available) -> t
      | Some (t, _, _, Unavailable) ->
-         error env id.at "M0025" "unavailable variable %s" id.it
-     | None -> error env id.at "M0026" "unbound variable %s" id.it
+       error env id.at "M0025" "unavailable variable %s" id.it
+     | None ->
+       error env id.at "M0026" "unbound variable %s%a%s" id.it
+         display_vals env.vals
+         (suggest "variable" id.it (T.Env.keys env.vals))
     )
   | DotH (path', id) ->
     let s, fs = check_obj_path env path' in
@@ -408,8 +485,14 @@ and check_obj_path' env path : T.typ =
       error env id.at "M0027" "cannot infer type of forward field reference %s" id.it
     | t -> t
     | exception Invalid_argument _ ->
-      error env id.at "M0028" "field %s does not exist in type%a"
-        id.it display_typ_expand (T.Obj (s, fs))
+      error env id.at "M0028" "field %s does not exist in %a%s"
+        id.it
+        display_obj (s, fs)
+        (suggest "field" id.it
+          (List.filter_map
+            (function
+              {T.typ=T.Typ _;_} -> None
+            | {T.lab;_} -> Some lab) fs))
 
 let rec check_typ_path env path : T.con =
   let c = check_typ_path' env path in
@@ -422,7 +505,10 @@ and check_typ_path' env path : T.con =
     use_identifier env id.it;
     (match T.Env.find_opt id.it env.typs with
     | Some c -> c
-    | None -> error env id.at "M0029" "unbound type %s" id.it
+    | None ->
+      error env id.at "M0029" "unbound type %s%a%s" id.it
+        display_typs env.typs
+        (suggest "type" id.it (T.Env.keys env.typs))
     )
   | DotH (path', id) ->
     let s, fs = check_obj_path env path' in
@@ -431,9 +517,12 @@ and check_typ_path' env path : T.con =
         check_deprecation env path.at "type field" id.it (T.lookup_typ_deprecation id.it fs);
         c
       | exception Invalid_argument _ ->
-        error env id.at "M0030" "type field %s does not exist in type%a"
+        error env id.at "M0030" "type field %s does not exist in type%a%s"
           id.it display_typ_expand (T.Obj (s, fs))
-
+          (suggest "type field" id.it
+             (List.filter_map
+               (function { T.lab; T.typ=T.Typ _;_ } -> Some lab
+               |  _ -> None) fs))
 
 (* Type helpers *)
 
@@ -1160,7 +1249,9 @@ and infer_exp'' env exp : T.typ =
       else t
     | Some (t, _, _, Available) -> id.note <- (if T.is_mut t then Var else Const); t
     | None ->
-      error env id.at "M0057" "unbound variable %s" id.it
+      error env id.at "M0057" "unbound variable %s%a%s" id.it
+        display_vals env.vals
+        (suggest "variable" id.it (T.Env.keys env.vals))
     )
   | LitE lit ->
     T.Prim (infer_lit env lit exp.at)
@@ -1376,7 +1467,7 @@ and infer_exp'' env exp : T.typ =
     T.(glb t_base (Obj (Object, sort T.compare_field fts)))
   | DotE (exp1, id) ->
     let t1 = infer_exp_promote env exp1 in
-    let _s, tfs =
+    let s, tfs =
       try T.as_obj_sub [id.it] t1 with Invalid_argument _ ->
       try array_obj (T.as_array_sub t1) with Invalid_argument _ ->
       try blob_obj (T.as_prim_sub T.Blob t1) with Invalid_argument _ ->
@@ -1396,9 +1487,14 @@ and infer_exp'' env exp : T.typ =
       t
     | exception Invalid_argument _ ->
       error env exp1.at "M0072"
-        "field %s does not exist in type%a"
+        "field %s does not exist in %a%s"
         id.it
-        display_typ_expand t1
+        display_obj (s, tfs)
+        (suggest "field" id.it
+          (List.filter_map
+             (function
+               { T.typ=T.Typ _;_} -> None
+             | {T.lab;_} -> Some lab) tfs))
     )
   | AssignE (exp1, exp2) ->
     if not env.pre then begin
@@ -1596,7 +1692,9 @@ and infer_exp'' env exp : T.typ =
         match String.split_on_char ' ' id.it with
         | ["continue"; name] -> name
         | _ -> id.it
-      in local_error env id.at "M0083" "unbound label %s" name
+      in local_error env id.at "M0083" "unbound label %s%a%s" name
+         display_labs env.labs
+         (suggest "label" id.it (T.Env.keys env.labs))
     );
     T.Non
   | RetE exp1 ->

diff --git a/src/mo_types/type.ml b/src/mo_types/type.ml
@@ -1379,7 +1379,7 @@ let motoko_runtime_information_type =
 
 let motoko_runtime_information_fld =
   { lab = "__motoko_runtime_information";
-    typ = Func(Shared Query, Promises, [scope_bind], [], 
+    typ = Func(Shared Query, Promises, [scope_bind], [],
       [ motoko_runtime_information_type ]);
     src = empty_src;
   }

diff --git a/src/prelude/internals.mo b/src/prelude/internals.mo
@@ -400,13 +400,13 @@ module @ManagementCanister = {
   };
 };
 
-type WasmMemoryPersistence = {
+type @WasmMemoryPersistence = {
   #Keep;
   #Replace;
 };
 
-type UpgradeOptions = { 
-  wasm_memory_persistence: ?WasmMemoryPersistence;
+type @UpgradeOptions = {
+  wasm_memory_persistence: ?@WasmMemoryPersistence;
 };
 
 let @ic00 = actor "aaaaa-aa" :
@@ -416,10 +416,10 @@ let @ic00 = actor "aaaaa-aa" :
       sender_canister_version : ?Nat64
     } -> async { canister_id : Principal };
     install_code : {
-      mode : { 
-        #install; 
-        #reinstall; 
-        #upgrade : ?UpgradeOptions;
+      mode : {
+        #install;
+        #reinstall;
+        #upgrade : ?@UpgradeOptions;
       };
       canister_id : Principal;
       wasm_module : @ManagementCanister.wasm_module;
@@ -434,7 +434,7 @@ func @install_actor_helper(
       #install : Principal;
       #reinstall : actor {} ;
       #upgrade : actor {} ;
-      #upgrade_with_persistence : { wasm_memory_persistence: WasmMemoryPersistence; canister: actor {} };
+      #upgrade_with_persistence : { wasm_memory_persistence: @WasmMemoryPersistence; canister: actor {} };
     },
     enhanced_orthogonal_persistence : Bool,
     wasm_module : Blob,
@@ -459,10 +459,10 @@ func @install_actor_helper(
         (#reinstall, (prim "principalOfActor" : (actor {}) -> Principal) actor1)
       };
       case (#upgrade actor2) {
-        let wasm_memory_persistence = if enhanced_orthogonal_persistence { 
-          ?(#Keep) 
-        } else { 
-          null 
+        let wasm_memory_persistence = if enhanced_orthogonal_persistence {
+          ?(#Keep)
+        } else {
+          null
         };
         let upgradeOptions = {
           wasm_memory_persistence;

diff --git a/test/fail/ok/M0028.tc.ok b/test/fail/ok/M0028.tc.ok
@@ -1,2 +1,2 @@
-M0028.mo:2.11-2.12: type error [M0028], field Y does not exist in type
+M0028.mo:2.11-2.12: type error [M0028], field Y does not exist in type:
   module {}
diff --git a/test/fail/ok/bad-type-comp.tc.ok b/test/fail/ok/bad-type-comp.tc.ok
@@ -8,7 +8,9 @@ cannot produce expected type
   module {type T = Null}
 bad-type-comp.mo:7.73-7.74: type error [M0018], duplicate type field name T in object type
 bad-type-comp.mo:11.15-11.16: type error [M0029], unbound type T
+Did you mean type Text?
 bad-type-comp.mo:17.15-17.16: type error [M0029], unbound type U
 bad-type-comp.mo:24.17-24.27: type error [M0137], type T = A__9 references type parameter A__9 from an outer scope
 bad-type-comp.mo:29.23-29.33: type error [M0137], type T = A__10 references type parameter A__10 from an outer scope
 bad-type-comp.mo:35.25-35.26: type error [M0029], unbound type T
+Did you mean type Text?
diff --git a/test/fail/ok/modexp1.tc.ok b/test/fail/ok/modexp1.tc.ok
@@ -1,2 +1,2 @@
-modexp1.mo:8.13-8.14: type error [M0072], field g does not exist in type
+modexp1.mo:8.13-8.14: type error [M0072], field g does not exist in type:
   module {f : () -> ()}
diff --git a/test/fail/ok/pretty.tc.ok b/test/fail/ok/pretty.tc.ok
@@ -45,7 +45,7 @@ pretty.mo:28.23-28.24: type error [M0096], expression of type
   }
 cannot produce expected type
   Nat
-pretty.mo:40.1-40.12: type error [M0072], field foo does not exist in type
+pretty.mo:40.1-40.12: type error [M0072], field foo does not exist in type:
   module {
     bar1 : Nat;
     bar2 : Nat;

diff --git a/test/fail/ok/suggest-label-ai.tc.ok b/test/fail/ok/suggest-label-ai.tc.ok
@@ -0,0 +1,3 @@
+suggest-label-ai.mo:3.9-3.11: type error [M0083], unbound label fo in label environment:
+  {foo : ()}
+The label fo is not available. Try something else?