dashboard: Decode requests (#5621)

* bus-client: Connect before publish

The client instance in events/defaultSubscribers/dashboard
isn't actually firing anything at the moment. This is fix.

* dashboard: WorkflowCompileResult message
- Create message
- Fire it from events and catch it in subscriber in frontend

* dashboard: Add @truffle/decoder dep

* dashboard: Add @truffle/codec dep

* dashboard: Add object-inspect deps

* dashboard: Fix bad `tsx` file extension

* dashboard: Add @truffle/compile-common dep

* dashboard: Fix naming mistake

* dashboard: Update @truffle/ deps

* dashboard: Add object-hash deps

* dashboard: Add idb dep

* dashboard: Simplify bus client init logic

* dashboard: State improvements
- Better name for message bus client.
- Remove host and port info.

* dashboard: Extract bus client init logic into its own function

* dashboard: IndexedDB setup
Init with schema and keep in state

* dashboard: Migrate from react-scripts to webpack + babel
Moving away from react-scripts@5 is necessary for decoder integration

* dashboard: Add to and organize existing dev server scripts
For some reason webpack cli cannot `webpack serve` even though webpack-dev-server
is installed. The workaround is to serve programmatically.

* dashboard: Webpack build directly into destination during `build` command
No need to copy

* dashboard: Remove cpy-cli devDep

* dashboard: Add polyfills necessary for web3

* dashboard: Add eslint devDep

* dashboard: Add eslint-webpack-plugin devDep

* dashboard: Configure eslint webpack plugin

* dashboard: Upgrade truffle namespace deps to latest

* dashboard: Remove unnecessary nested build directory level

* dashboard: Some webpack config for niceness

* dashboard: Decoder setup
Init with persisted compilations and keep in state

* dashboard: Upgrade concurrently devDep

* dashboard: Use concurrently programmatically to start dev servers

* dashboard: Webpack log progress

* dashboard: Use only message lifecycle to handle message
No need to pass provider around

* dashboard: Handle WorkflowCompileResultMessage (partial impl.)
Keep track of compilations
 - Best effort: Check uniqueness using sha1; I haven't verified that it always works.
Re-init decoder when new compilations come
 - This will change when #5602 is resolved,
   so decoder is only initialized once. Warning: That way we are mutating state, which is
   forbidden, but it saves memory. I will experiment, my hope is that since it doesn't
   directly touch the dom it's OK.
 * Edit: Instead of the decoder staying in state, keep it in a ref. Let something else
   trigger rerender.

This is partial because nothing makes it to db yet.

Also slightly reworked how reducer returns new state when handling bus messages. It's easier to follow now.

* dashboard: Init decoder before bus client
It'd be bad if WorkflowCompileResultMessage comes in before decoder is initialized

* dashboard: Decode and render in `<RPC />`

* dashboard: Remove decoding from `<RPC />` state
There is no reference to it. This will be added back when codec components come.

* dashboard: Fix initDecoder bad logic
Should set at index, not push to compilations array

* dashboard: Handle WorkflowCompileResultMessage (finished impl.)
Persist to and prune compilations from db.

* dashboard: Remove unnecessary label

* dashboard: Move log and debug message handling outside of reducer

* dashboard: Tighten SetDecoderAction data property type

* dashboard: Add terser-webpack-plugin devDep

* dashboard: Properly mute console via webpack
Remove ce6265f hack

* dashboard: Update comment on batch insert optimization
With batch insert, the write to idb is a larger transaction and has more gotchas.
A large atomic transaction means it'd be bad if one small thing goes wrong and
the rest of the write fails with it.

It gets more complicated when dashboard runs in multiple contexts (e.g. >1 tabs).
You'd want to pick one context to do the work. This picking process involves using
BroadcastChannel to determine which context.

I think this should stay simple.

* dashboard: Wrap db insert in try-catch

* dashboard: Webpack only minimize for build
Dev server recompile cannot afford to minimize, takes too long.

* dashboard: Add warning sign when decoding fails.

* dashboard: Change labels

* dashboard: Move eth_signTransaction to array of unsupported methods
eth_signTransaction isn't supported by MetaMask

* dashboard: Only try to decode for eth_sendTransaction

* dashboard: Better `<RPC />` decoding-related logic and display

* dashboard: Use icon when showing decode fail

* dashboard: Notification when transaction decoding fails / recovers

* dashboard: Fix idb import

* dashboard: Decoder use `window.ethereum`

* dashboard: Confirm provider messages using `window.ethereum`

* dashboard: Remove unused provider from state and remove references to it

* dashboard: Remove unnecessary async/await

* dashboard: Remove unnecessary @ts-ignore comment

* dashboard: Prevent prism from crashing when decoding isn't ready

* dashboard: Support decoding personal_sign requests
Also extract decoding logic and notifications into utils.

* dashboard: Remove unnecessary variable to indicate `<Details />` view

* dashboard: Tighten prop type for inspected decoding

* dashboard: Change code highlight language depending on request method
Also extract:
- `<DecodedParams />` and `<RawParams />` out of `<Expanded />`
- Some prism styles into `<Global />`

* dashboard: Nest some styles in `<Global />`

* dashboard: Change some variable / type names from upper to camel / pascal case

* dashboard: Minor fix mute error regex pattern

* dashboard: Add missing Buffer import

* dashboard: Support decoding eth_signTypedData_v3 + eth_signTypedData_v4
Organized notifications

* dashboard: Fix code highlight for decoded typed data

* dashboard: Use latest @truffle deps

* dashboard: Stdin accept ^C to stop dev servers

* dashboard: Hide decode notification in effect clean-up

* dashboard: Move hide notification logic into its own effect
Otherwise when the decoder recovers, the success notification won't show

* dashboard: Add missing dependency in `<RPC />` effects

* workflow-compile: Wait for "compile:succeed" event handlers

* dashboard: Webpack config use `nosources-source-map`
Align with truffle build source map config

* dashboard: Switch babel-loader out for ts-loader

* dashboard: Remove unused babel deps and config
The remaining @babel/* deps are required by eslint-config-react-app

* dashboard: Generalize WorkflowCompileResultMessage to be CliEventMessage

* bus-common: Remove package.json empty dependencies field

* dashboard: Remove \.jsx? check in webpack config

* dashboard: Update some deps

* dashboard: Change @truffle/compile-common to be devDep

* dashboard: Pass around decoding object
Instead of the resulting string from inspect

* bus-common: Improve CliEventMessage type
Default to unknown payload data type

* dashboard: Update to latest @truffle deps

* dashboard: Create and use Decoding type

* dashboard: Extract getHighlightLanguageForRpcMethod

* bus-common: Add generic to isCliEventMessage

* bus-common: Remove CliEventMessage default generic param

packages/dashboard-message-bus-client/lib/client.ts

@@ -74,6 +74,8 @@ export class DashboardMessageBusClient {
     const { type, payload } = options;
     let message = createMessage(type, payload);
     try {
+      await this.ready();
+
       const lifecycle = new PublishMessageLifecycle({
         message,
         connection: this._publishConnection

packages/dashboard-message-bus-common/lib/messages.ts

@@ -9,6 +9,20 @@ export interface Response {
   payload: any;
 }
 
+export type CliEventMessageType = "cli-event";
+export const cliEventMessageType = "cli-event";
+/**
+ * Message to inform subscribers of Truffle CLI event data.
+ * The message payload label helps identify the type of event data.
+ */
+export interface CliEventMessage<T> extends Message {
+  type: CliEventMessageType;
+  payload: {
+    label: string;
+    data: T;
+  };
+}
+
 export type DashboardProviderMessageType = "provider";
 export const dashboardProviderMessageType = "provider";
 /**
@@ -27,7 +41,6 @@ export interface DashboardProviderMessage extends Message {
 
 export type LogMessageType = "log";
 export const logMessageType = "log";
-
 /**
  * Message intended to log messages across the message bus.
  * The message payload includes a "debug" namespace as well as a message.
@@ -43,7 +56,6 @@ export interface LogMessage extends Message {
 
 export type DebugMessageType = "debug";
 export const debugMessageType = "debug";
-
 /**
  * Message to log in Dashboard browser console
  */
@@ -66,6 +78,12 @@ export interface InvalidateMessage extends Message {
   payload: number;
 }
 
+export const isCliEventMessage = <T>(
+  message: Message
+): message is CliEventMessage<T> => {
+  return message.type === cliEventMessageType;
+};
+
 export const isDashboardProviderMessage = (
   message: Message
 ): message is DashboardProviderMessage => {
@@ -87,6 +105,7 @@ export const isInvalidateMessage = (
 };
 
 export type MessageType =
+  | CliEventMessage<unknown>
   | DashboardProviderMessageType
   | LogMessageType
   | DebugMessageType

packages/dashboard/dev-server/all.ts

@@ -0,0 +1,48 @@
+import concurrently from "concurrently";
+
+function notice(text: string) {
+  console.log(`\x1b[34m\x1b[1m${text}\x1b[0m`);
+}
+
+async function startServers() {
+  notice("Press ^C or Q to exit\nStarting servers...");
+
+  const { commands, result } = concurrently(
+    [
+      {
+        command: "yarn start:server",
+        name: "dashboard",
+        prefixColor: "bgMagenta.bold"
+      },
+      {
+        command: "yarn start:react",
+        name: "webpack",
+        prefixColor: "cyan.bgBlack.bold"
+      }
+    ],
+    {
+      killOthers: ["success", "failure"],
+      handleInput: true
+    }
+  );
+
+  const kill = () => commands.forEach(command => void command.kill("SIGINT"));
+
+  process.stdin.setRawMode(true);
+  process.stdin.on("data", async data => {
+    const isControlC = data.equals(Buffer.from("\x03"));
+    const isQ = /^q$/i.test(data.toString().trim());
+
+    if (isControlC || isQ) {
+      notice("Stopping servers...");
+      kill();
+      try {
+        await result;
+      } catch (err) {
+        /* We don't really care. */
+      }
+    }
+  });
+}
+
+startServers();

packages/dashboard/bin/start-dev-server.ts → packages/dashboard/dev-server/dashboard.ts

@@ -7,6 +7,13 @@ const options = {
   rpc: true,
   autoOpen: false
 };
-
 const dashboardServer = new DashboardServer(options);
-dashboardServer.start();
+
+async function startServer() {
+  await dashboardServer.start();
+  console.log(
+    `Dashboard server running at http://${options.host}:${options.port}`
+  );
+}
+
+startServer();

packages/dashboard/dev-server/webpack.ts

@@ -0,0 +1,12 @@
+import Webpack from "webpack";
+import WebpackDevServer from "webpack-dev-server";
+import webpackConfig from "../webpack.config";
+
+const compiler = Webpack(webpackConfig);
+const server = new WebpackDevServer(webpackConfig.devServer, compiler);
+
+async function startServer() {
+  await server.start();
+}
+
+startServer();

packages/dashboard/lib/DashboardServer.ts

@@ -70,12 +70,7 @@ export class DashboardServer {
     this.rpc = options.rpc ?? true;
     this.verbose = options.verbose ?? false;
     this.autoOpen = options.autoOpen ?? true;
-    this.frontendPath = path.join(
-      __dirname,
-      ".",
-      "dashboard-frontend",
-      "build"
-    );
+    this.frontendPath = path.join(__dirname, "dashboard-frontend");
 
     this.boundTerminateListener = () => this.stop();
   }

packages/dashboard/package.json

@@ -21,15 +21,13 @@
     "dist/lib"
   ],
   "scripts": {
-    "build": "yarn build:react && yarn build:server && yarn copy:react",
-    "build:react": "react-scripts build",
-    "build:server": "tsc -p tsconfig.server.json",
-    "copy:react": "cpy build/** dist/lib/dashboard-frontend/ --parents --dot",
+    "build": "yarn build:server && yarn build:react --output-path ./dist/lib/dashboard-frontend",
+    "build:react": "cross-env NODE_ENV=\"production\" webpack",
+    "build:server": "tsc --project tsconfig.server.json",
     "prepare": "yarn build",
-    "start": "concurrently --kill-others \"yarn start:dev-server\" \"NODE_ENV=\"development\" yarn start:react\"",
-    "start:dev-server": "ts-node --project tsconfig.server.json bin/start-dev-server.ts",
-    "start:prebuilt": "node dist/bin/start-dev-server.js",
-    "start:react": "react-scripts start",
+    "start": "ts-node --project tsconfig.server.json ./dev-server/all.ts",
+    "start:react": "ts-node --project tsconfig.server.json ./dev-server/webpack.ts",
+    "start:server": "ts-node --project tsconfig.server.json ./dev-server/dashboard.ts",
     "test": "jest --passWithNoTests"
   },
   "types": "dist/lib/index.d.ts",
@@ -39,42 +37,62 @@
     "@mantine/hooks": "^5.0.0",
     "@mantine/notifications": "^5.0.0",
     "@mantine/prism": "^5.0.0",
+    "@truffle/codec": "^0.14.9",
     "@truffle/dashboard-message-bus": "^0.1.8",
     "@truffle/dashboard-message-bus-client": "^0.1.8",
     "@truffle/dashboard-message-bus-common": "^0.1.4",
+    "@truffle/decoder": "^5.3.0",
     "cors": "^2.8.5",
     "debug": "^4.3.2",
     "ethers": "^5.6.9",
     "express": "^4.17.1",
     "get-port": "^5.1.1",
+    "idb": "^7.1.1",
     "isomorphic-ws": "^4.0.1",
+    "object-hash": "^3.0.0",
+    "object-inspect": "^1.12.2",
     "open": "^8.4.0",
     "react": "^18.2.0",
     "react-dom": "^18.2.0",
     "react-feather": "^2.0.10",
     "react-router-dom": "^6.3.0",
-    "react-scripts": "^5.0.1",
     "wagmi": "^0.6.3",
     "ws": "^7.2.0"
   },
   "devDependencies": {
-    "@babel/core": "^7.18.10",
+    "@babel/core": "^7.20.2",
     "@babel/plugin-syntax-flow": "^7.18.6",
-    "@babel/plugin-transform-react-jsx": "^7.18.10",
+    "@babel/plugin-transform-react-jsx": "^7.19.0",
+    "@truffle/compile-common": "^0.9.0",
     "@types/cors": "^2.8.12",
-    "@types/express": "^4.17.13",
+    "@types/express": "^4.17.14",
     "@types/jest": "27.4.1",
     "@types/node": "~12.12.0",
-    "@types/react": "^18.0.15",
-    "@types/react-dom": "^18.0.6",
+    "@types/object-hash": "^2.2.1",
+    "@types/object-inspect": "^1.8.1",
+    "@types/react": "^18.0.25",
+    "@types/react-dom": "^18.0.9",
     "@types/ws": "^7.2.0",
-    "concurrently": "^6.5.1",
-    "cpy-cli": "^3.1.1",
+    "concurrently": "^7.5.0",
     "cross-env": "^7.0.3",
+    "crypto-browserify": "^3.12.0",
+    "eslint": "^8.25.0",
     "eslint-config-react-app": "^7.0.1",
+    "eslint-webpack-plugin": "^3.2.0",
+    "html-webpack-plugin": "^5.5.0",
+    "os-browserify": "^0.3.0",
+    "path-browserify": "^1.0.1",
+    "process": "^0.11.10",
+    "stream-browserify": "^3.0.0",
+    "terser-webpack-plugin": "^5.3.6",
     "ts-jest": "29.0.3",
+    "ts-loader": "^9.4.1",
     "ts-node": "10.7.0",
-    "typescript": "^4.7.4"
+    "typescript": "^4.7.4",
+    "vm-browserify": "^1.1.2",
+    "webpack": "^5.74.0",
+    "webpack-cli": "^4.10.0",
+    "webpack-dev-server": "^4.11.1"
   },
   "publishConfig": {
     "access": "public"

packages/dashboard/public/index.html

@@ -1,40 +1,20 @@
 <!DOCTYPE html>
 <html lang="en">
-  <head>
-    <meta charset="utf-8" />
-    <link rel="icon" href="%PUBLIC_URL%/favicon.ico" />
-    <meta name="viewport" content="width=device-width, initial-scale=1" />
-    <meta name="theme-color" content="#000000" />
-    <meta name="description" content="Truffle Dashboard" />
-    <link rel="apple-touch-icon" href="%PUBLIC_URL%/logo192.png" />
-    <!--
-      manifest.json provides metadata used when your web app is installed on a
-      user's mobile device or desktop. See https://developers.google.com/web/fundamentals/web-app-manifest/
-    -->
-    <link rel="manifest" href="%PUBLIC_URL%/manifest.json" />
-    <!--
-      Notice the use of %PUBLIC_URL% in the tags above.
-      It will be replaced with the URL of the `public` folder during the build.
-      Only files inside the `public` folder can be referenced from the HTML.
 
-      Unlike "/favicon.ico" or "favicon.ico", "%PUBLIC_URL%/favicon.ico" will
-      work correctly both with client-side routing and a non-root public URL.
-      Learn how to configure a non-root public URL by running `npm run build`.
-    -->
-    <title>Truffle Dashboard</title>
-  </head>
-  <body>
-    <noscript>You need to enable JavaScript to run this app.</noscript>
-    <div id="root"></div>
-    <!--
-      This HTML file is a template.
-      If you open it directly in the browser, you will see an empty page.
+<head>
+  <meta charset="utf-8" />
+  <link rel="icon" href="favicon.ico" />
+  <meta name="viewport" content="width=device-width, initial-scale=1" />
+  <meta name="description" content="Truffle Dashboard" />
+  <title>Truffle Dashboard</title>
+</head>
 
-      You can add webfonts, meta tags, or analytics to this file.
-      The build step will place the bundled scripts into the <body> tag.
+<body>
+  <div id="root"></div>
+  <noscript>
+    JavaScript is disabled in your browser.
+    Please consider enabling it to run this app.
+  </noscript>
+</body>
 
-      To begin the development, run `npm start` or `yarn start`.
-      To create a production bundle, use `npm run build` or `yarn build`.
-    -->
-  </body>
 </html>

packages/dashboard/src/App.tsx

@@ -1,6 +1,7 @@
 // Wrappers
 import MantineWrapper from "src/components/wrappers/MantineWrapper";
 import ColorSchemeWrapper from "src/components/wrappers/ColorSchemeWrapper";
+import { NotificationsProvider } from "@mantine/notifications";
 import WagmiWrapper from "src/components/wrappers/WagmiWrapper";
 import { DashProvider } from "src/contexts/DashContext";
 // Router
@@ -17,22 +18,24 @@ function App(): JSX.Element {
       <ColorSchemeWrapper>
         <MantineWrapper>
           <MantineGlobal /> {/* Set global styles */}
-          <WagmiWrapper>
-            <DashProvider>
-              <BrowserRouter>
-                <Routes>
-                  <Route path="/" element={<Layout />}>
-                    <Route index element={<Navigate to="/rpcs" replace />} />
-                    <Route path="rpcs" element={<RPCs />} />
-                  </Route>
-                  {process.env.NODE_ENV === "development" && (
-                    <Route path="colors" element={<Palette />} />
-                  )}
-                  <Route path="*" element={<Navigate to="/" replace />} />
-                </Routes>
-              </BrowserRouter>
-            </DashProvider>
-          </WagmiWrapper>
+          <NotificationsProvider limit={5}>
+            <WagmiWrapper>
+              <DashProvider>
+                <BrowserRouter>
+                  <Routes>
+                    <Route path="/" element={<Layout />}>
+                      <Route index element={<Navigate to="/rpcs" replace />} />
+                      <Route path="rpcs" element={<RPCs />} />
+                    </Route>
+                    {process.env.NODE_ENV === "development" && (
+                      <Route path="colors" element={<Palette />} />
+                    )}
+                    <Route path="*" element={<Navigate to="/" replace />} />
+                  </Routes>
+                </BrowserRouter>
+              </DashProvider>
+            </WagmiWrapper>
+          </NotificationsProvider>
         </MantineWrapper>
       </ColorSchemeWrapper>
     </div>