squash commits

packages/render-fidelity-tools/test/README.md

@@ -0,0 +1,62 @@
+# Fidelity Testing
+
+## Crafting new test scenarios
+
+As far as `<model-viewer>` is concerned, a scenario is just an HTML file that
+incorporates `<model-viewer>` with a given configuration. This HTML file will
+be loaded by a browser at test time and used to produce a screenshot for
+comparison to any configured reference renders.
+
+The steps to produce a new scenario are:
+
+ 1. Acquire or produce a glTF that is representative of what the scenario should
+    be testing.
+      - If applicable, also produce a corresponding USDZ of that model
+ 2. Pick a descriptive name for the scenario (for example, `alpha-blend-litmus`)
+ 3. Create a sub-directory of the [`test/fidelity`](./) folder with the same
+    name as your scenario. This will be the scenario's directory.
+ 4. Add any relevant screenshots of reference viewers to the scenario's
+    directory
+      - See above in this document for advice on creating reference viewer
+        screenshots
+ 5. Add an `index.html` document containing a configured `<model-viewer>`, and
+    with CSS margins removed from the `<body>`.
+      - Currently, screenshots are taken in an area measured from the top-left of
+        the document, so any margin will most likely lead to undesired results
+ 6. Modify the configuration at [`test/fidelity/config.json`](./config.json) to
+    include an item under `scenarios` that corresponds to your newly created
+    scenario
+      - Be sure to specify the dimensions of the test, the set of references
+        (called "goldens") and the thresholds to test at (0, 1 and 10 are
+        recommended).
+
+
+On execution, a stringyfied scenario configuration will automatically be attached to `args`, the list of arguments to the `executable`. Please note the additional `outputFile` property.
+
+Example configuration:
+```json
+{ 
+  "scenario": {
+    "lighting": "../../../shared-assets/environments/lightroom_14b.hdr",
+    "dimensions": {
+      "width": 768,
+      "height": 450
+    },
+    "target": {
+      "x": 0,
+      "y": 0.3,
+      "z": 0
+    },
+    "orbit": {
+      "theta": 0,
+      "phi": 90,
+      "radius": 1
+    },
+    "verticalFoV": 45,
+    "renderSkybox": False,
+    "name": "khronos-SheenChair",
+    "model": "../../../shared-assets/models/glTF-Sample-Assets/Models/SheenChair/glTF-Binary/SheenChair.glb"
+  },
+  "outputFile": "./test/goldens/khronos-SheenChair/stellar-golden.png"
+}
+```

packages/render-fidelity-tools/test/ic_modelviewer.svg

@@ -0,0 +1,13 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<svg width="24px" height="24px" viewBox="0 0 24 24" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
+    <!-- Generator: Sketch 54.1 (76490) - https://sketchapp.com -->
+    <title>Artboard Copy 7</title>
+    <desc>Created with Sketch.</desc>
+    <g id="Artboard-Copy-7" stroke="none" stroke-width="1" fill="none" fill-rule="evenodd">
+        <g id="Group">
+            <rect id="Rectangle" x="0" y="0" width="24" height="24"></rect>
+            <path d="M12,21 C16.9705627,21 21,16.9705627 21,12 C21,7.02943725 16.9705627,3 12,3 C7.02943725,3 3,7.02943725 3,12 C3,16.9705627 7.02943725,21 12,21 Z M12,23 C5.92486775,23 1,18.0751322 1,12 C1,5.92486775 5.92486775,1 12,1 C18.0751322,1 23,5.92486775 23,12 C23,18.0751322 18.0751322,23 12,23 Z" id="Oval-Copy" fill="#000000" fill-rule="nonzero"></path>
+            <path d="M15.267627,8.88656534 L12.0625,7.03608439 C12.0238249,7.01375529 11.9761751,7.01375529 11.9375,7.03608439 L8.74222777,8.88087567 L12.0098548,10.767441 L15.267627,8.88656534 Z M16.330127,10.5825318 L13,12.5051815 L13,16.4226497 L16.267627,14.5360844 C16.3063022,14.5137553 16.330127,14.4724894 16.330127,14.4278312 L16.330127,10.5825318 Z M11,16.4226497 L11,12.4938022 L7.66987298,10.5711524 L7.66987298,14.4278312 C7.66987298,14.4724894 7.69369785,14.5137553 7.73237298,14.5360844 L11,16.4226497 L11,16.4226497 Z M17.267627,7.7318648 C17.9251043,8.11145949 18.330127,8.81297941 18.330127,9.57216878 L18.330127,14.4278312 C18.330127,15.1870206 17.9251043,15.8885405 17.267627,16.2681352 L13.0625,18.6959664 C12.4050227,19.0755611 11.5949773,19.0755611 10.9375,18.6959664 L6.73237298,16.2681352 C6.07489569,15.8885405 5.66987298,15.1870206 5.66987298,14.4278312 L5.66987298,9.57216878 C5.66987298,8.81297941 6.07489569,8.11145949 6.73237298,7.7318648 L10.9375,5.30403358 C11.5949773,4.92443889 12.4050227,4.92443889 13.0625,5.30403358 L17.267627,7.7318648 Z" id="Shape" fill="#000000" fill-rule="nonzero"></path>
+        </g>
+    </g>
+</svg>

packages/render-fidelity-tools/test/renderers/model-viewer/index.html

@@ -0,0 +1,85 @@
+<!--
+/* @license
+ * Copyright 2019 Google Inc. All Rights Reserved.
+ * Licensed under the Apache License, Version 2.0 (the 'License');
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an 'AS IS' BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+-->
+<!doctype html>
+<html>
+<head>
+  <meta charset="UTF-8">
+  <meta name="viewport" content="width=device-width, minimum-scale=1.0, initial-scale=1.0, user-scalable=yes">
+  <title>&lt;model-viewer&gt; Fidelity Tests</title>
+
+  <style>
+    body { margin: 0; }
+    model-viewer { 
+      --progress-bar-color: transparent;
+    }
+  </style>
+</head>
+<body>
+  <renderer-harness id="harness">
+    <span slot="title">&lt;model-viewer&gt;</span>
+    <model-viewer id="modelViewer"
+        slot="renderer"
+        camera-controls
+        interaction-prompt="none">
+    </model-viewer>
+  </renderer-harness>
+  <script type="module" src="../../../dist/renderer-harness.js"></script>
+  <script type="module" src="../../../../../node_modules/@google/model-viewer/dist/model-viewer.min.js"></script>
+  <script type="module">
+    customElements.get("model-viewer").minimumRenderScale = 1;
+  </script>
+  <script>
+    const harness = document.getElementById('harness');
+    const modelViewer = document.getElementById('modelViewer');
+    harness.addEventListener('scenario-change', () => {
+      const {scenario} = harness;
+      modelViewer.addEventListener('poster-dismissed', () => {
+        // Sometimes the screenshot is taken when the poster hasn't faded away. 
+        // This is a temporary workaround so that the screenshot is taken two frames after the poster-dismissed event fires.
+        // TODO(sun765): Remove this when the bug is fixed. 
+        requestAnimationFrame(()=>{
+          requestAnimationFrame(()=>{
+            requestAnimationFrame(()=>{
+              this.dispatchEvent(new CustomEvent('model-ready'));
+            })
+          })
+        });
+      }, {once: true});
+      
+      modelViewer.style.width = `${scenario.dimensions.width}px`;
+      modelViewer.style.height = `${scenario.dimensions.height}px`;
+
+      modelViewer.style.backgroundColor = "rgba(255,255,255,0)";
+
+      modelViewer.src = scenario.model;
+
+      modelViewer.environmentImage = scenario.lighting;
+      modelViewer.skyboxImage = scenario.renderSkybox? scenario.lighting: null;
+
+      const {theta, phi, radius} = scenario.orbit;
+      modelViewer.minCameraOrbit = `auto auto ${radius}m`;
+      modelViewer.maxCameraOrbit = `auto auto ${radius}m`;
+      modelViewer.cameraOrbit = `${theta}deg ${phi}deg ${radius}m`;
+      const {x, y, z} = scenario.target;
+      modelViewer.cameraTarget = `${x}m ${y}m ${z}m`;
+      modelViewer.maxFieldOfView = `${scenario.verticalFoV}deg`;
+      modelViewer.minFieldOfView = `${scenario.verticalFoV}deg`;
+      modelViewer.fieldOfView = `${scenario.verticalFoV}deg`;
+    });
+  </script>
+</body>
+</html>

packages/render-fidelity-tools/test/results-viewer.html

@@ -0,0 +1,123 @@
+<!--
+/* @license
+ * Copyright 2018 Google Inc. All Rights Reserved.
+ * Licensed under the Apache License, Version 2.0 (the 'License');
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an 'AS IS' BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+-->
+<!DOCTYPE html>
+<html lang="en">
+<head>
+  <title>&lt;model-viewer&gt; - Fidelity Test Results</title>
+  <meta charset="utf-8">
+  <meta name="viewport" content="width=device-width, initial-scale=1">
+  <link rel="shortcut icon" type="image/png" href="../shared-assets/icons/favicon.png"/>
+
+  <script defer src="https://web3dsurvey.com/collector.js"></script>
+  <script>
+    window.ga=window.ga||function(){(ga.q=ga.q||[]).push(arguments)};ga.l=+new Date;
+    ga('create', 'UA-169901325-1', { 'storage': 'none' });
+    ga('set', 'referrer', document.referrer.split('?')[0]);
+    ga('set', 'anonymizeIp', true);
+    ga('send', 'pageview');
+  </script>
+  <script async src='https://www.google-analytics.com/analytics.js'></script>
+<style>
+  @import url('https://fonts.googleapis.com/css?family=Rubik:400,500');
+  body {
+    margin: 1em;
+    background-color: #f7f7f7;
+    font-family: 'Rubik', sans-serif;
+    font-size: 16px;
+    line-height: 24px;
+    color: rgba(0,0,0,.87);
+    font-weight: 400;
+    -webkit-font-smoothing: antialiased;
+  }
+
+  p {
+    max-width: 1000px;
+    text-align: left;
+  }
+  .icon-modelviewer-black {
+    background-image: url(../../assets/ic_modelviewer.svg);
+  }
+  .icon-button {
+    margin-left: -4px;
+    margin-right: 8px;
+    width: 34px;
+    height: 34px;
+    background-size: 34px;
+  }
+  .inner-home {
+    display: flex;
+    align-items: center;
+    font-size: 1.1em;
+    text-decoration: none;
+  }
+  .home {
+    padding: 20px;
+    overflow: auto;
+    white-space: nowrap;
+  }
+  .lockup {
+    display: flex;
+    align-items: center;
+    margin-bottom: 6px;
+    color: rgba(0,0,0,.87);
+  }
+  .attribute {
+    white-space: pre-wrap !important;
+    font-family: 'Roboto Mono', monospace;
+    color: black;
+  }
+  .attribute:hover {
+    text-decoration: underline;
+    color: #444444;
+  }
+</style>
+</head>
+<body>
+  <div class="home lockup">
+    <a href="../" class="sidebar-mv inner-home">
+      <div class="icon-button icon-modelviewer-black inner-home"></div>
+      <div class="inner-home"><span class="attribute">&lt;model-viewer&gt;</span></div>
+    </a>
+  </div>
+  <div align="center">
+    <h2>Render Fidelity Testing</h2>
+
+    <p>The first set of scenarios are for demonstrating material capabilities,
+    so these are primarily fidelity tests. Following these are simpler cases
+    that test glTF spec conformance, but are not particularly interesting for
+    fidelity. At the bottom are the Furnace Test and the Directional Light Test.
+    The Furnace Test checks energy conservation and the correct result is the
+    whole 7x7 grid of metal-rough spheres being invisible (indistinguishable
+    from the background). The top (metal) and bottom (dielectric) rows are
+    passing, while the mixtures lose some energy. The Directional Light Test
+    checks the worst-case scenario for environmental lighting approximations:
+    the case where a single bright pixel represents a directional light.</p>
+
+    <p>We show a simple logarithmic metric on an average of the pixel
+    differences, excluding the transparent background. This is <b>not</b> a
+    perceptual metric, so take it with a grain of salt. It is mostly there to
+    help us identify regressions.</p>
+
+    <p><a href="https://policies.google.com/privacy">Privacy</a></p>
+  </div>
+
+  <image-comparison-app src="./results/config.json"></image-comparison-app>
+
+  <script type="module" src="../dist/image-comparison-app.js"></script>
+
+</body>
+</html>