squash commits
This commit is contained in:
MrPlatnum committed
commit
0f545c5e4d
611 files changed
+272565
No files matched your search
@@ -0,0 +1,6 @@
|
||||
shared-assets
|
||||
node_modules
|
||||
test/renderers/filament/ktx
|
||||
test/renderers/filament/cmgen
|
||||
test/results
|
||||
test/renderers/dspbr-pt/shader
|
||||
@@ -0,0 +1,202 @@
|
||||
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
APPENDIX: How to apply the Apache License to your work.
|
||||
|
||||
To apply the Apache License to your work, attach the following
|
||||
boilerplate notice, with the fields enclosed by brackets "[]"
|
||||
replaced with your own identifying information. (Don't include
|
||||
the brackets!) The text should be enclosed in the appropriate
|
||||
comment syntax for the file format. We also recommend that a
|
||||
file or class name and description of purpose be included on the
|
||||
same "printed page" as the copyright notice for easier
|
||||
identification within third-party archives.
|
||||
|
||||
Copyright [yyyy] [name of copyright owner]
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
@@ -0,0 +1,70 @@
|
||||
# Render fidelity tools
|
||||
|
||||
This package contains tools and testing processes that enable the
|
||||
`<model-viewer>` project to ensure high-fidelity 3D model rendering.
|
||||
|
||||
If you are looking for the `<model-viewer>` web component, please refer to the
|
||||
[`<model-viewer>` package](../model-viewer) and also the live docs and examples
|
||||
on [modelviewer.dev](https://modelviewer.dev).
|
||||
|
||||
## Development
|
||||
|
||||
To get started, follow the instructions in [the main README.md file](../../README.md).
|
||||
|
||||
In order to run `npm run render-goldens`, you must have `imagemagick` installed.
|
||||
|
||||
On macOS, you can install `imagemagick` with homebrew: `brew install imagemagick`
|
||||
|
||||
On Debian/Ubuntu/Pop!_OS you can install `imagemagick` with `apt`: `sudo apt install imagemagick`
|
||||
|
||||
The following commands are available when developing modelviewer.dev:
|
||||
|
||||
Command | Description
|
||||
------------------------------- | -----------
|
||||
`npm run build` | Build artifacts required to run testing and review tools
|
||||
`npm run clean` | Deletes all build artifacts
|
||||
`npm run test` | Run a render fidelity check comparing `<model-viewer>` to other renderers
|
||||
`npm run render-goldens` | Render the "golden" images for all renderers
|
||||
|
||||
### Fidelity Test CLI
|
||||
|
||||
For `npm run test`, we support these command line options:
|
||||
|
||||
Command | Default | Description
|
||||
--------------------|----------- | -----------
|
||||
-c, --config | | Path to configuration json.
|
||||
-s, --scenario | | Limit to specific scenarios. This now also allows you to specify multiple scenarios in the whitelist. You can use a full name or a partial name of scenarios and it will match against all that contain that scenario substring.
|
||||
-p, --port | 9040 | Port for web server.
|
||||
-d, --dry-run | false | Lists which images would be rendered but doesn't render. Useful when trying to figure out which tests will run given that command line.
|
||||
-q, --quiet | false | Hide the puppeteer controlled browser. This can allow you to work on the computer while it is running in the background.
|
||||
|
||||
To run a subset of scenarios do something like this:
|
||||
|
||||
```
|
||||
% npm run test -- --scenario=texture --quiet
|
||||
```
|
||||
|
||||
*NOTE: In the above example, the `--` is required to separate the arguments to `npm run test` from the arguments to the script itself.*
|
||||
|
||||
After running `npm run test`, you can look at the results by starting a local web server (e.g. `npx http-server`) in this folder, then opening `test/results-viewer.html` in a browser.
|
||||
|
||||
### Render Goldens CLI
|
||||
|
||||
For `npm run render-goldens`, we support these command line options:
|
||||
|
||||
Command | Default | Description
|
||||
--------------------|----------- | -----------
|
||||
-c, --config | | Path to configuration json.
|
||||
-s, --scenario | | Limit to specific scenarios. This now also allows you to specify multiple scenarios in the whitelist. You can use a full name or a partial name of scenarios and it will match against all that contain that scenario substring.
|
||||
-p, --port | 9040 | Port for web server.
|
||||
-m, --missing-only | false | Only render if an output image is missing. Very useful when adding new tests.
|
||||
-d, --dry-run | false | Lists which images would be rendered but doesn't render. Useful when trying to figure out which tests will run given that command line.
|
||||
-q, --quiet | false | Hide the puppeteer controlled browser. This can allow you to work on the computer while it is running in the background.
|
||||
|
||||
To run a subset of renders or scenarios do something like this:
|
||||
|
||||
```
|
||||
% npm run render-goldens -- --scenario=clearcoat -q
|
||||
```
|
||||
|
||||
*NOTE: In the above example, the `--` is required to separate the arguments to `npm run render-goldens` from the arguments to the script itself.*
|
||||
@@ -0,0 +1,50 @@
|
||||
{
|
||||
"name": "@google/model-viewer-render-fidelity-tools",
|
||||
"type": "module",
|
||||
"private": true,
|
||||
"version": "0.0.1",
|
||||
"description": "<model-viewer> rendering fidelity test suite and tools",
|
||||
"scripts": {
|
||||
"test": "node --experimental-modules ./lib/workflows/test-fidelity.js --config=./test/config.json",
|
||||
"test:ci": "echo \"CI fidelity tests have a separate run\" && exit 0",
|
||||
"render-goldens": "node --experimental-modules ./lib/workflows/render-goldens.js --config=./test/config.json",
|
||||
"build": "tsc && rollup -c",
|
||||
"prepare": "if [ ! -L './shared-assets' ]; then ln -s ../shared-assets ./shared-assets; fi",
|
||||
"clean": "rm -rf ./dist ./lib ./test/results ./test/renderers/filament/{cmgen,ktx}"
|
||||
},
|
||||
"contributors": [
|
||||
"Chris Joel <[email protected]>",
|
||||
"Emmett Lalish <[email protected]>",
|
||||
"Jaixin Sun <[email protected]>"
|
||||
],
|
||||
"license": "Apache-2.0",
|
||||
"engines": {
|
||||
"node": ">=12.0.0"
|
||||
},
|
||||
"dependencies": {
|
||||
"@actions/core": "^1.11.1",
|
||||
"@google/model-viewer": "^4.1.0",
|
||||
"@polymer/paper-button": "^3.0.1",
|
||||
"@polymer/paper-radio-group": "^3.0.1",
|
||||
"@polymer/paper-slider": "^3.0.1",
|
||||
"@types/http-server": "^0.12.4",
|
||||
"@types/pngjs": "^6.0.5",
|
||||
"http-server": "^14.1.1",
|
||||
"lit": "^3.2.1",
|
||||
"mkdirp": "^3.0.1",
|
||||
"pngjs": "^7.0.0",
|
||||
"puppeteer": "^24.4.0",
|
||||
"rimraf": "^6.0.1",
|
||||
"yargs": "^17.7.2"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@rollup/plugin-commonjs": "^28.0.3",
|
||||
"@rollup/plugin-node-resolve": "^16.0.0",
|
||||
"@rollup/plugin-replace": "^6.0.2",
|
||||
"@types/yargs": "^17.0.33",
|
||||
"polymer-build": "^3.1.4",
|
||||
"rollup": "^4.35.0",
|
||||
"rollup-plugin-external-globals": "^0.13.0",
|
||||
"typescript": "5.8.2"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,64 @@
|
||||
/*
|
||||
* Copyright 2020 Google LLC. All Rights Reserved.
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
import commonjs from '@rollup/plugin-commonjs';
|
||||
import {nodeResolve as resolve} from '@rollup/plugin-node-resolve';
|
||||
import replace from '@rollup/plugin-replace';
|
||||
import {basename} from 'path';
|
||||
import externalGlobals from 'rollup-plugin-external-globals';
|
||||
|
||||
const onwarn = (warning, warn) => {
|
||||
// Suppress non-actionable warning caused by TypeScript boilerplate:
|
||||
if (warning.code !== 'THIS_IS_UNDEFINED') {
|
||||
warn(warning);
|
||||
}
|
||||
};
|
||||
|
||||
const plugins = [
|
||||
commonjs(),
|
||||
resolve({preferBuiltins: true, browser: true}),
|
||||
replace({'Reflect.decorate': 'undefined', preventAssignment: true}),
|
||||
externalGlobals({filament: 'Filament'})
|
||||
];
|
||||
|
||||
const buildTarget = (input, outputFormat) => ({
|
||||
input,
|
||||
output: {
|
||||
file: `./dist/${basename(input)}`,
|
||||
sourcemap: true,
|
||||
format: outputFormat,
|
||||
name: outputFormat === 'esm' ?
|
||||
undefined :
|
||||
basename(input)
|
||||
.split('-')
|
||||
.map(
|
||||
string =>
|
||||
`${string.slice(0, 1).toUpperCase()}${string.slice(1)}`)
|
||||
.join('')
|
||||
},
|
||||
watch: {
|
||||
include: '{lib/**,lib/third_party/**}',
|
||||
},
|
||||
plugins,
|
||||
onwarn
|
||||
});
|
||||
|
||||
const outputOptions = [
|
||||
buildTarget('./lib/components/image-comparison-app.js', 'esm'),
|
||||
buildTarget('./lib/components/renderer-harness.js', 'esm'),
|
||||
buildTarget('./lib/image-comparison-worker.js', 'iife')
|
||||
];
|
||||
|
||||
export default outputOptions;
|
||||
@@ -0,0 +1,35 @@
|
||||
#!/bin/bash
|
||||
|
||||
##
|
||||
# Copyright 2018 Google Inc. All Rights Reserved.
|
||||
# Licensed under the Apache License, Version 2.0 (the 'License');
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an 'AS IS' BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
##
|
||||
|
||||
if [ "${TEST_TYPE}" = "fidelity" ]; then
|
||||
set -x
|
||||
|
||||
git checkout origin/gh-pages -- ./fidelity
|
||||
LAST_RESULT="`tail -n 1 ./fidelity/manifest`"
|
||||
|
||||
set -e
|
||||
|
||||
if [ -z "$LAST_RESULT" ]; then
|
||||
echo "No previous fidelity results available for comparison!";
|
||||
exit 0;
|
||||
fi
|
||||
|
||||
node ./scripts/compare-fidelity-results.js "./test/fidelity/results" "./fidelity/$LAST_RESULT"
|
||||
|
||||
set +e
|
||||
set +x
|
||||
fi
|
||||
@@ -0,0 +1,47 @@
|
||||
#!/bin/bash
|
||||
|
||||
##
|
||||
# Copyright 2018 Google Inc. All Rights Reserved.
|
||||
# Licensed under the Apache License, Version 2.0 (the 'License');
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an 'AS IS' BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
##
|
||||
|
||||
set -x
|
||||
set -e
|
||||
|
||||
if [ -z "$TMPDIR" ]; then
|
||||
TMPDIR="/tmp/";
|
||||
fi
|
||||
|
||||
REF=$1
|
||||
CLONE_DIR="${TMPDIR}model-viewer-master"
|
||||
CURRENT_DIR="`pwd`"
|
||||
|
||||
if [ -d "$CLONE_DIR" ]; then
|
||||
rm -rf "$CLONE_DIR"
|
||||
fi
|
||||
|
||||
git clone --branch $REF ./ "$CLONE_DIR"
|
||||
cd "$CLONE_DIR"
|
||||
|
||||
npm install
|
||||
npm run build
|
||||
|
||||
set +e
|
||||
|
||||
npm run check-fidelity
|
||||
|
||||
cd "$CURRENT_DIR"
|
||||
|
||||
node ./scripts/compare-fidelity-results.js "$CURRENT_DIR/test/fidelity/results" "$CLONE_DIR/test/fidelity/results"
|
||||
|
||||
set +x
|
||||
@@ -0,0 +1,71 @@
|
||||
#!/bin/bash
|
||||
|
||||
##
|
||||
# Copyright 2018 Google Inc. All Rights Reserved.
|
||||
# Licensed under the Apache License, Version 2.0 (the 'License');
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an 'AS IS' BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
##
|
||||
|
||||
##
|
||||
# This script updates the screenshots of the third-party renderers that we are
|
||||
# able to compare to <model-viewer>`. If there are any changes to the
|
||||
# screenshots, the script then generates a branch and commits the screenshots to
|
||||
# it. Finally, it opens a PR against the main project on Github.
|
||||
#
|
||||
# NOTE: This script is only intended to be run as part of automation
|
||||
# infrastructure. It will probably work anywhere with the right environment
|
||||
# variables set but please use it with caution!
|
||||
##
|
||||
|
||||
if [ -z "$MODEL_VIEWER_CHECKOUT_DIRECTORY" ] || [ -z "$GIT_USER_EMAIL" ] || [ -z "$GIT_USER_NAME" ] || [ -z "$GITHUB_USER" ] || [ -z "$GITHUB_ACCESS_TOKEN" ] || [ -z "$MODEL_VIEWER_BASE_BRANCH" ]; then
|
||||
echo '🚨 Some variables needed to create screenshot update PR were not set!'
|
||||
exit 1;
|
||||
fi
|
||||
|
||||
pushd $MODEL_VIEWER_CHECKOUT_DIRECTORY
|
||||
|
||||
# Take a new set of screenshots
|
||||
npm run update-screenshots
|
||||
|
||||
FIDELITY_SCREENSHOT_PATH=$MODEL_VIEWER_CHECKOUT_DIRECTORY/test/fidelity
|
||||
NUMBER_CHANGED_SCREENSHOTS=`git ls-files -m $FIDELITY_SCREENSHOT_PATH | wc -l`
|
||||
|
||||
# If any screenshots changed, we will be creating a branch and crafting a PR
|
||||
if [ "$NUMBER_CHANGED_SCREENSHOTS" -gt "0" ]; then
|
||||
UPDATES_BRANCH="screenshot-updates-`TZ='America/Los_Angeles' date +"%m-%d-%y"`"
|
||||
BRANCH_SHA=`git rev-parse --verify $UPDATES_BRANCH`
|
||||
|
||||
# If we already ran today, we already have a branch by this name so delete it
|
||||
if [ ! -z "$BRANCH_SHA" ]; then
|
||||
git branch -d $UPDATES_BRANCH
|
||||
fi
|
||||
|
||||
git checkout -b $UPDATES_BRANCH
|
||||
# Only add changes in the fidelity testing subtree
|
||||
git add $FIDELITY_SCREENSHOT_PATH
|
||||
|
||||
COMMIT_MESSAGE="🤖 Render fidelity updates as of `TZ='America/Los_Angeles' date`"
|
||||
git config user.name "$GIT_USER_NAME"
|
||||
git config user.email "$GIT_USER_EMAIL"
|
||||
git commit -m "$COMMIT_MESSAGE"
|
||||
git push -f origin $UPDATES_BRANCH
|
||||
|
||||
PULL_REQUEST_BODY="**NOTE**: This PR was automatically generated. It contains updated render results from third-party renderers for use in our fidelity tests. Please remember to take the following steps when reviewing:\n\n - Review the changed images and take note of any patterns (same renderer, same model etc)\n - Review fidelity test results in the CI build and take note of any reported changes\n\nTo read more about \`<model-viewer>\` fidelity testing and learn how to update these screenshots yourself, check out [this explainer](https://github.com/google/model-viewer/blob/master/test/fidelity/README.md).\n\nPlease refer to this [startup script](https://github.com/google/model-viewer/blob/master/scripts/render-vm-startup-script.sh) for technical details related to how this PR was produced.";
|
||||
|
||||
# Create the PR via Github's API
|
||||
curl --user "$GITHUB_USER:$GITHUB_ACCESS_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
--data "{ \"title\": \"$COMMIT_MESSAGE\", \"head\": \"$UPDATES_BRANCH\", \"base\": \"$MODEL_VIEWER_BASE_BRANCH\", \"body\": \"$PULL_REQUEST_BODY\", \"maintainer_can_modify\": true }" \
|
||||
https://api.github.com/repos/googlewebcomponents/model-viewer/pulls
|
||||
fi
|
||||
|
||||
popd
|
||||
@@ -0,0 +1,80 @@
|
||||
#!/bin/bash
|
||||
|
||||
##
|
||||
# Copyright 2018 Google Inc. All Rights Reserved.
|
||||
# Licensed under the Apache License, Version 2.0 (the 'License');
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an 'AS IS' BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
##
|
||||
|
||||
##
|
||||
# Provision an Ubuntu 18.04 VM for performing automated tasks on behalf of the
|
||||
# <model-viewer> project. VM is assumed to have an attached NVIDIA GPU. This
|
||||
# script doesn't guarantee idempotency, but it *should* be safe to run multiple
|
||||
# times.
|
||||
##
|
||||
|
||||
if [ -z "$MODEL_VIEWER_CHECKOUT_DIRECTORY" ]; then
|
||||
MODEL_VIEWER_CHECKOUT_DIRECTORY=`pwd`
|
||||
fi
|
||||
|
||||
# Set the DISPLAY so that headless windows launch correctly
|
||||
export DISPLAY=:0
|
||||
|
||||
# Set up build toolchain apt repos
|
||||
sudo -E apt-add-repository -y "ppa:ubuntu-toolchain-r/test"
|
||||
wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key|sudo apt-key add -
|
||||
echo "deb http://apt.llvm.org/xenial/ llvm-toolchain-xenial-7 main" | sudo tee -a /etc/apt/sources.list >/dev/null
|
||||
|
||||
# Check for CUDA and try to install NVIDIA drivers
|
||||
if ! dpkg-query -W cuda-10-0; then
|
||||
curl -O http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-repo-ubuntu1804_10.0.130-1_amd64.deb
|
||||
sudo dpkg -i ./cuda-repo-ubuntu1804_10.0.130-1_amd64.deb
|
||||
sudo apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/7fa2af80.pub
|
||||
sudo apt-get update
|
||||
sudo apt-get install cuda-10-0 -y
|
||||
fi
|
||||
|
||||
# Enable persistence mode
|
||||
sudo nvidia-smi -pm 1
|
||||
|
||||
# Install build chain, graphics and X.org packages
|
||||
sudo apt-get install -y \
|
||||
xserver-xorg \
|
||||
freeglut3-dev \
|
||||
mesa-common-dev \
|
||||
mesa-utils \
|
||||
libxmu-dev \
|
||||
libglu1-mesa-dev \
|
||||
libc++-7-dev \
|
||||
libc++abi-7-dev \
|
||||
libstdc++-7-dev \
|
||||
clang-7 \
|
||||
cmake \
|
||||
ninja-build \
|
||||
libxi-dev
|
||||
|
||||
# Start X.org as a daemon
|
||||
sudo /sbin/start-stop-daemon --start --quiet --pidfile /tmp/custom_xvfb_99.pid --make-pidfile --background --exec /usr/bin/X -- :0
|
||||
sleep 3 # Give X.org a few seconds to start up...
|
||||
|
||||
# Set up NVM and Node.js/NPM
|
||||
curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.34.0/install.sh | bash
|
||||
export NVM_DIR="$HOME/.nvm"
|
||||
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"
|
||||
nvm install --lts
|
||||
|
||||
pushd $MODEL_VIEWER_CHECKOUT_DIRECTORY
|
||||
|
||||
# Install NPM dependencies
|
||||
npm install
|
||||
|
||||
popd
|
||||
@@ -0,0 +1,52 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
##
|
||||
# Copyright 2018 Google Inc. All Rights Reserved.
|
||||
# Licensed under the Apache License, Version 2.0 (the 'License');
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an 'AS IS' BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
##
|
||||
|
||||
set -x
|
||||
set -e
|
||||
|
||||
if [ -z "$TRAVIS_TAG" ]; then
|
||||
echo '$TRAVIS_TAG is not set!';
|
||||
exit 1;
|
||||
fi
|
||||
|
||||
TEST_RESULTS="./test/fidelity/results"
|
||||
FIDELITY_ROOT="./fidelity"
|
||||
RECORD_MANIFEST="$FIDELITY_ROOT/manifest"
|
||||
OUTPUT_DIRECTORY="$FIDELITY_ROOT/$TRAVIS_TAG"
|
||||
|
||||
set +e
|
||||
|
||||
git checkout origin/gh-pages -- ./fidelity
|
||||
|
||||
set -e
|
||||
|
||||
if [[ ! -d $FIDELITY_ROOT ]]; then
|
||||
mkdir -p $FIDELITY_ROOT;
|
||||
fi
|
||||
|
||||
if [[ -d $OUTPUT_DIRECTORY ]]; then
|
||||
rm -rf $OUTPUT_DIRECTORY;
|
||||
fi
|
||||
|
||||
cp -r $TEST_RESULTS $OUTPUT_DIRECTORY
|
||||
|
||||
# All directories in the root are recorded in a file for later reference
|
||||
# by the frontend of the Github Pages deployment:
|
||||
ls -l $FIDELITY_ROOT | egrep '^d' | awk '{ print $9 }' > $RECORD_MANIFEST
|
||||
|
||||
set +x
|
||||
set +e
|
||||
@@ -0,0 +1,63 @@
|
||||
#!/bin/bash
|
||||
|
||||
##
|
||||
# Copyright 2018 Google Inc. All Rights Reserved.
|
||||
# Licensed under the Apache License, Version 2.0 (the 'License');
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an 'AS IS' BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
##
|
||||
|
||||
##
|
||||
# Render VM Startup Script
|
||||
#
|
||||
# This script is intended to be used in an Ubuntu 18.04 cloud VM with an NVIDIA
|
||||
# Tesla GPU. It is designed to run on startup, provision the VM if necessary,
|
||||
# collect screenshots and create a PR on Github if there are any changes.
|
||||
#
|
||||
# NOTE: In order to use this script as a startup script in your own VM,
|
||||
# uncomment the variables below and fill them in as appropriate, or else make
|
||||
# sure the environment variables are set before this script is run.
|
||||
##
|
||||
|
||||
# These values should definitely be customized:
|
||||
# export [email protected]
|
||||
# export GIT_USER_NAME="User Name"
|
||||
# export GITHUB_USER=example
|
||||
# export GITHUB_ACCESS_TOKEN=your-github-access-token-here
|
||||
|
||||
# These values may not need to be customized:
|
||||
export MODEL_VIEWER_REPOSITORY=https://$GITHUB_USER:$GITHUB_ACCESS_TOKEN@github.com/google/model-viewer.git
|
||||
export MODEL_VIEWER_CHECKOUT_DIRECTORY=$HOME/model-viewer
|
||||
export MODEL_VIEWER_BASE_BRANCH=master
|
||||
|
||||
##
|
||||
# NOTE: No need to modify below this line when configuring for deployment
|
||||
##
|
||||
|
||||
# Clone the repo if necessary
|
||||
if [ ! -d ./model-viewer ]; then
|
||||
git clone $MODEL_VIEWER_REPOSITORY $MODEL_VIEWER_CHECKOUT_DIRECTORY
|
||||
fi
|
||||
|
||||
pushd $MODEL_VIEWER_CHECKOUT_DIRECTORY
|
||||
|
||||
# Update <model-viewer> repo from the origin and reset the tracked branch
|
||||
git fetch origin
|
||||
git add .
|
||||
git reset --hard origin/$MODEL_VIEWER_BASE_BRANCH
|
||||
|
||||
popd
|
||||
|
||||
# Invoke relevant startup sub-scripts
|
||||
source $MODEL_VIEWER_CHECKOUT_DIRECTORY/scripts/provision-ubuntu-xenial-vm.sh
|
||||
source $MODEL_VIEWER_CHECKOUT_DIRECTORY/scripts/create-screenshot-update-pr.sh
|
||||
|
||||
sudo poweroff
|
||||
@@ -0,0 +1,391 @@
|
||||
/* @license
|
||||
* Copyright 2019 Google LLC. All Rights Reserved.
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
import {promises as fs} from 'fs';
|
||||
import {mkdirp} from 'mkdirp';
|
||||
import {join, resolve} from 'path';
|
||||
import pngjs from 'pngjs';
|
||||
import puppeteer, {Browser} from 'puppeteer';
|
||||
|
||||
import {DEVICE_PIXEL_RATIO, Dimensions, FIDELITY_TEST_THRESHOLD, FidelityRegressionResults, GoldenConfig, ImageComparator, ImageComparisonAnalysis, ImageComparisonConfig, ScenarioConfig, toDecibel} from './common.js';
|
||||
import {ConfigReader} from './config-reader.js';
|
||||
|
||||
|
||||
const $configReader = Symbol('configReader');
|
||||
|
||||
export type AnalysisResults = Array<ImageComparisonAnalysis>;
|
||||
|
||||
export interface ScenarioRecord extends ScenarioConfig {
|
||||
analysisResults: AnalysisResults;
|
||||
}
|
||||
|
||||
export class ArtifactCreator {
|
||||
private[$configReader]: ConfigReader = new ConfigReader(this.config);
|
||||
private browser: Browser|null = null;
|
||||
|
||||
constructor(
|
||||
protected config: ImageComparisonConfig, protected rootDirectory: string,
|
||||
protected baseUrl: string) {
|
||||
console.log('🌈 Preparing to capture screenshots for fidelity comparison');
|
||||
}
|
||||
|
||||
async close() {
|
||||
if (this.browser != null) {
|
||||
await this.browser.close();
|
||||
this.browser = null;
|
||||
}
|
||||
}
|
||||
protected get outputDirectory(): string {
|
||||
return join(resolve(this.rootDirectory), 'results');
|
||||
}
|
||||
|
||||
protected get goldens(): Array<GoldenConfig> {
|
||||
return this.config.renderers.map(
|
||||
renderer => ({...renderer, file: `${renderer.name}-golden.png`}));
|
||||
}
|
||||
|
||||
async compareRenderers(scenario: ScenarioConfig, renderer: string) {
|
||||
const analysisResults: AnalysisResults = [];
|
||||
const {rootDirectory, outputDirectory, goldens} = this;
|
||||
const {name: scenarioName, exclude, dimensions} = scenario;
|
||||
|
||||
// skip if this renderer is excluded from this scenario
|
||||
if (exclude != null && exclude.includes(renderer)) {
|
||||
return;
|
||||
}
|
||||
|
||||
console.log(
|
||||
`Start to compare ${renderer}'s golden with other renderers' goldens:`);
|
||||
|
||||
const rendererIndex = 0;
|
||||
const rendererGoldenPath = join(
|
||||
rootDirectory, 'goldens', scenarioName, goldens[rendererIndex].file);
|
||||
let rendererGolden;
|
||||
try {
|
||||
rendererGolden = await fs.readFile(rendererGoldenPath);
|
||||
} catch (error) {
|
||||
throw new Error(`❌ Failed to read ${renderer}'s ${
|
||||
scenarioName} golden! Error message: ${error.message}`);
|
||||
}
|
||||
|
||||
// save goldens images to result folder which will be used to show in the
|
||||
// result-viewer page.
|
||||
await fs.writeFile(
|
||||
join(outputDirectory, scenarioName, goldens[rendererIndex].file),
|
||||
rendererGolden);
|
||||
|
||||
const rendererGoldenImage =
|
||||
new Uint8ClampedArray(pngjs.PNG.sync.read(rendererGolden).data);
|
||||
|
||||
for (const golden of goldens) {
|
||||
if (golden.name === renderer ||
|
||||
(exclude != null && exclude.includes(golden.name))) {
|
||||
continue;
|
||||
}
|
||||
|
||||
console.log(`\n🔍 Comparing ${renderer} to ${golden.description}`);
|
||||
|
||||
const candidateGoldenPath =
|
||||
join(rootDirectory, 'goldens', scenarioName, golden.file);
|
||||
let candidateGolden;
|
||||
try {
|
||||
candidateGolden = await fs.readFile(candidateGoldenPath);
|
||||
} catch (error) {
|
||||
throw new Error(`❌ Failed to read ${golden.name}'s ${
|
||||
scenarioName} golden! Error message: ${error.message}`);
|
||||
}
|
||||
|
||||
// save goldens images to result folder which will be used to show in the
|
||||
// result-viewer page.
|
||||
await fs.writeFile(
|
||||
join(outputDirectory, scenarioName, golden.file), candidateGolden);
|
||||
|
||||
const candidateGoldenImage =
|
||||
new Uint8ClampedArray(pngjs.PNG.sync.read(candidateGolden).data);
|
||||
const analysisResult = await this.analyze(
|
||||
rendererGoldenImage, candidateGoldenImage, dimensions);
|
||||
analysisResults.push(analysisResult);
|
||||
}
|
||||
const scenarioRecord = {analysisResults, scenario};
|
||||
|
||||
console.log(`\n💾 Recording analysis`);
|
||||
await fs.writeFile(
|
||||
join(outputDirectory, scenarioName, 'analysis.json'),
|
||||
JSON.stringify(scenarioRecord));
|
||||
}
|
||||
|
||||
async captureAndAnalyzeScreenshot(
|
||||
scenario: ScenarioConfig, renderer: string,
|
||||
quiet: boolean = false): Promise<ImageComparisonAnalysis> {
|
||||
const {rootDirectory, goldens} = this;
|
||||
const {name: scenarioName, dimensions, exclude} = scenario;
|
||||
|
||||
console.log(`start compare ${renderer}'s golden with ${
|
||||
renderer}'s screenshot generated from fidelity test:`);
|
||||
|
||||
let screenshot: Uint8Array|undefined;
|
||||
try {
|
||||
// set the output path to an empty string to tell puppeteer to not save
|
||||
// the screenshot image
|
||||
screenshot = await this.captureScreenshot(
|
||||
renderer, scenarioName, dimensions, '', 60, quiet);
|
||||
} catch (error) {
|
||||
if (this.browser != null) {
|
||||
await this.browser.close();
|
||||
this.browser = null;
|
||||
}
|
||||
throw new Error(`❌ Failed to capture ${renderer}'s screenshot of ${
|
||||
scenarioName}. Error message: ${error.message}`);
|
||||
}
|
||||
|
||||
if (screenshot == null) {
|
||||
throw new Error(`❌ ${renderer}'s screenshot of ${
|
||||
scenarioName} is not captured correctly (value is null).`);
|
||||
}
|
||||
const buffer = Buffer.from(screenshot);
|
||||
const screenshotImage =
|
||||
new Uint8ClampedArray(pngjs.PNG.sync.read(buffer).data);
|
||||
|
||||
const rendererIndex = 0;
|
||||
const rendererGoldenPath = join(
|
||||
rootDirectory, 'goldens', scenarioName, goldens[rendererIndex].file);
|
||||
let rendererGolden;
|
||||
try {
|
||||
rendererGolden = await fs.readFile(rendererGoldenPath);
|
||||
} catch (error) {
|
||||
throw new Error(`❌ Failed to read ${renderer}'s ${
|
||||
scenarioName} golden! Error message: ${error.message}`);
|
||||
}
|
||||
const rendererGoldenImage =
|
||||
new Uint8ClampedArray(pngjs.PNG.sync.read(rendererGolden).data);
|
||||
|
||||
const result =
|
||||
await this.analyze(screenshotImage, rendererGoldenImage, dimensions);
|
||||
|
||||
const rmsInDb = toDecibel(result.rmsDistanceRatio);
|
||||
|
||||
// the rmsInDb is negative, and the less negative means the less closer the
|
||||
// two images are
|
||||
if (rmsInDb > FIDELITY_TEST_THRESHOLD) {
|
||||
if (exclude?.includes(renderer)) {
|
||||
console.log(`❌ Skipped! Senario name: ${
|
||||
scenario.name}, rms distance ratio: ${rmsInDb.toFixed(2)} dB.`);
|
||||
} else {
|
||||
throw new Error(`❌ Senarios name: ${
|
||||
scenario.name}, rms distance ratio: ${rmsInDb.toFixed(2)} dB.`);
|
||||
}
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
async fidelityTest(
|
||||
scenarioWhitelist: Set<string>|null = null, renderer: string,
|
||||
dryRun: boolean = false, quiet: boolean = false) {
|
||||
const {scenarios} = this.config;
|
||||
const {outputDirectory} = this;
|
||||
const analyzedScenarios: Array<ScenarioConfig> = [];
|
||||
const fidelityRegressionResults:
|
||||
FidelityRegressionResults = {results: [], errors: [], warnings: []};
|
||||
|
||||
const compareRenderersErrors: Array<string> = [];
|
||||
|
||||
for (const scenarioBase of scenarios) {
|
||||
const scenarioName = scenarioBase.name;
|
||||
const scenario = this[$configReader].scenarioConfig(scenarioName)!;
|
||||
|
||||
// skip if white lists exists and this scenario isn't in it
|
||||
if (scenarioWhitelist != null && !scenarioWhitelist.has(scenarioName)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
// skip if this renderer is excluded from this scenario
|
||||
if (scenario.exclude != null && scenario.exclude.includes(renderer)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
console.log(`\n🎨 Scenario: ${scenarioName}`);
|
||||
|
||||
const scenarioOutputDirectory = join(outputDirectory, scenarioName);
|
||||
mkdirp.sync(scenarioOutputDirectory);
|
||||
|
||||
try {
|
||||
await this.compareRenderers(scenario, renderer);
|
||||
} catch (error) {
|
||||
const errorMessage =
|
||||
`❌Fail to compare ${renderer} with other renderers of scenario ${
|
||||
scenarioName}. Error message: ${error.message}`;
|
||||
compareRenderersErrors.push(errorMessage);
|
||||
}
|
||||
|
||||
if (!dryRun) {
|
||||
try {
|
||||
const autoTestResult =
|
||||
await this.captureAndAnalyzeScreenshot(scenario, renderer, quiet);
|
||||
fidelityRegressionResults.results.push(autoTestResult);
|
||||
} catch (error) {
|
||||
const message = `❌Fail to analyze scenario :${
|
||||
scenarioName}! Error message: ${error.message}`;
|
||||
|
||||
fidelityRegressionResults.errors.push(message);
|
||||
}
|
||||
|
||||
analyzedScenarios.push(scenario);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
console.log('💾 Recording configuration');
|
||||
|
||||
const finalConfig: ImageComparisonConfig = Object.assign(
|
||||
{},
|
||||
this.config,
|
||||
{scenarios: analyzedScenarios, errors: compareRenderersErrors});
|
||||
|
||||
// ensure directory exists, this can happen if all scenarios are ignored
|
||||
await fs.mkdir(outputDirectory, {recursive: true});
|
||||
|
||||
await fs.writeFile(
|
||||
join(outputDirectory, 'config.json'), JSON.stringify(finalConfig));
|
||||
|
||||
await fs.writeFile(
|
||||
join(outputDirectory, 'fidelityRegressionResults.json'),
|
||||
JSON.stringify(fidelityRegressionResults));
|
||||
}
|
||||
|
||||
protected async analyze(
|
||||
candidateImage: Uint8ClampedArray, goldenImage: Uint8ClampedArray,
|
||||
dimensions: Dimensions): Promise<ImageComparisonAnalysis> {
|
||||
const imageDimensions = {
|
||||
width: dimensions.width * DEVICE_PIXEL_RATIO,
|
||||
height: dimensions.height * DEVICE_PIXEL_RATIO
|
||||
};
|
||||
const comparator =
|
||||
new ImageComparator(candidateImage, goldenImage, imageDimensions);
|
||||
|
||||
const {analysis} = comparator.analyze();
|
||||
const {rmsDistanceRatio} = analysis;
|
||||
console.log(`\n 📊 Decibels of root mean square color distance: ${
|
||||
(10 * Math.log10(rmsDistanceRatio)).toFixed(2)}`);
|
||||
|
||||
return analysis;
|
||||
}
|
||||
|
||||
|
||||
async captureScreenshot(
|
||||
renderer: string, scenarioName: string, dimensions: Dimensions,
|
||||
outputPath: string = join(this.outputDirectory, 'model-viewer.png'),
|
||||
maxTimeInSec: number = -1, quiet: boolean = false) {
|
||||
const scaledWidth = dimensions.width;
|
||||
const scaledHeight = dimensions.height;
|
||||
const rendererConfig = this[$configReader].rendererConfig(renderer);
|
||||
|
||||
if (rendererConfig == null) {
|
||||
console.log(`⚠️ Renderer "${
|
||||
renderer}" is not configured. Did you add it to the test config?`);
|
||||
return;
|
||||
}
|
||||
|
||||
if (this.browser == null) {
|
||||
console.log(`🚀 Launching browser`);
|
||||
// no-sandbox and disable-setuid-sandbox args to resolve puppeteer browser run error in fidelity tests
|
||||
this.browser = await puppeteer.launch({headless: quiet, args:['--no-sandbox', '--disable-setuid-sandbox']});
|
||||
}
|
||||
|
||||
const page = await this.browser.newPage();
|
||||
|
||||
const url = `${this.baseUrl}?hide-ui&config=../../config.json&scenario=${
|
||||
encodeURIComponent(scenarioName)}`;
|
||||
|
||||
await page.setViewport({
|
||||
width: scaledWidth,
|
||||
height: scaledHeight,
|
||||
deviceScaleFactor: DEVICE_PIXEL_RATIO
|
||||
});
|
||||
|
||||
page.on('error', (error: any) => {
|
||||
console.log(`🚨 ${error}`);
|
||||
});
|
||||
|
||||
page.on('console', async (message: any) => {
|
||||
const args =
|
||||
await Promise.all(message.args().map((arg: any) => arg.jsonValue()));
|
||||
|
||||
if (args.length) {
|
||||
console.log(`➡️`, ...args);
|
||||
}
|
||||
});
|
||||
|
||||
console.log(`🗺 Navigating to ${url}`);
|
||||
|
||||
await page.goto(url);
|
||||
|
||||
console.log(
|
||||
`🖌 Rendering ${scenarioName} with ${rendererConfig.description}`);
|
||||
|
||||
// NOTE: The function passed to page.evaluate is stringified and eval'd
|
||||
// in a browser context. Importantly, this implies that no external
|
||||
// variables are captured in its closure scope. TypeScript compiler
|
||||
// currently has no mechanism to detect this and will happily tell you
|
||||
// your code is correct when it isn't.
|
||||
const evaluateError = await page.evaluate(async (maxTimeInSec: number) => {
|
||||
const modelBecomesReady = new Promise<void>((resolve, reject) => {
|
||||
let timeout: NodeJS.Timeout;
|
||||
if (maxTimeInSec > 0) {
|
||||
timeout = setTimeout(() => {
|
||||
reject(new Error(
|
||||
`Stop capturing screenshot after ${maxTimeInSec} seconds`));
|
||||
}, maxTimeInSec * 1000);
|
||||
}
|
||||
|
||||
self.addEventListener('model-ready', () => {
|
||||
if (maxTimeInSec > 0) {
|
||||
clearTimeout(timeout);
|
||||
}
|
||||
resolve();
|
||||
}, {once: true});
|
||||
});
|
||||
|
||||
try {
|
||||
await modelBecomesReady;
|
||||
return null;
|
||||
} catch (error) {
|
||||
return error.message;
|
||||
}
|
||||
}, maxTimeInSec);
|
||||
|
||||
if (evaluateError) {
|
||||
console.log(evaluateError);
|
||||
await this.browser.close();
|
||||
this.browser = null;
|
||||
throw new Error(evaluateError);
|
||||
}
|
||||
|
||||
console.log(`🖼 Capturing screenshot`);
|
||||
|
||||
try {
|
||||
await fs.mkdir(this.outputDirectory);
|
||||
} catch (e) {
|
||||
// Ignored...
|
||||
}
|
||||
|
||||
const screenshot =
|
||||
await page.screenshot({path: outputPath as `${string}.png`, omitBackground: true});
|
||||
|
||||
page.close();
|
||||
|
||||
return screenshot;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,295 @@
|
||||
/* @license
|
||||
* Copyright 2019 Google LLC. All Rights Reserved.
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
import {colorDelta} from './third_party/pixelmatch/color-delta.js';
|
||||
|
||||
export interface OffscreenCanvas extends HTMLCanvasElement {}
|
||||
|
||||
export const COMPONENTS_PER_PIXEL: number = 4;
|
||||
|
||||
// 35215 is the maximum possible value for the YIQ difference metric
|
||||
// @see https://github.com/mapbox/pixelmatch/blob/master/index.js#L14
|
||||
// @see http://www.progmat.uaem.mx:8080/artVol2Num2/Articulo3Vol2Num2.pdf
|
||||
export const MAX_COLOR_DISTANCE: number = 35215;
|
||||
|
||||
export const DEVICE_PIXEL_RATIO: number = 2;
|
||||
|
||||
// use this threshold to do automatic fidelity test for model-viewer. any
|
||||
// scenario whose rms value (in dB) is bigger than the threshold will fail.
|
||||
export const FIDELITY_TEST_THRESHOLD: number = -22;
|
||||
|
||||
export interface FidelityRegressionResults {
|
||||
results: Array<ImageComparisonAnalysis>;
|
||||
warnings: Array<string>;
|
||||
errors: Array<string>;
|
||||
}
|
||||
|
||||
export interface ImageComparisonAnalysis {
|
||||
rmsDistanceRatio: number;
|
||||
}
|
||||
|
||||
export interface ImageComparisonResults {
|
||||
analysis: ImageComparisonAnalysis
|
||||
}
|
||||
|
||||
export interface Visuals {
|
||||
imageBuffers: {delta: ArrayBuffer|null; blackWhite: ArrayBuffer | null;};
|
||||
}
|
||||
|
||||
export interface ScenarioRecord {
|
||||
analysisResults: Array<ImageComparisonAnalysis>;
|
||||
scenarioConfig: ScenarioConfig;
|
||||
}
|
||||
|
||||
export interface ImageComparisonMessage {
|
||||
type: 'canvases-ready'|'images-assigned'|'threshold-changed'|
|
||||
'analysis-completed';
|
||||
}
|
||||
|
||||
export interface CanvasesReadyMessage extends ImageComparisonMessage {
|
||||
candidateCanvas: OffscreenCanvas;
|
||||
goldenCanvas: OffscreenCanvas;
|
||||
blackWhiteCanvas: OffscreenCanvas;
|
||||
deltaCanvas: OffscreenCanvas;
|
||||
}
|
||||
|
||||
export interface ImagesAssignedMessage extends ImageComparisonMessage {
|
||||
candidateImageBuffer: ArrayBuffer;
|
||||
goldenImageBuffer: ArrayBuffer;
|
||||
dimensions: Dimensions;
|
||||
}
|
||||
|
||||
export interface ThresholdChangedMessage extends ImageComparisonMessage {
|
||||
threshold: number;
|
||||
}
|
||||
|
||||
export interface AnalysisCompletedMessage extends ImageComparisonMessage {
|
||||
result: Visuals
|
||||
}
|
||||
|
||||
export interface Dimensions {
|
||||
width: number;
|
||||
height: number;
|
||||
}
|
||||
|
||||
export interface Pixel {
|
||||
x: number;
|
||||
y: number;
|
||||
}
|
||||
|
||||
export interface Rect extends Dimensions {
|
||||
x: number;
|
||||
y: number;
|
||||
}
|
||||
|
||||
|
||||
export interface ScenarioConfig {
|
||||
name: string;
|
||||
model: string;
|
||||
lighting: string;
|
||||
dimensions: Dimensions;
|
||||
target: {x: number, y: number, z: number};
|
||||
orbit: {theta: number, phi: number, radius: number};
|
||||
verticalFoV: number;
|
||||
exclude?: Array<string>;
|
||||
renderSkybox: boolean;
|
||||
pt?: {numSamples?: number};
|
||||
}
|
||||
|
||||
export interface RendererConfig {
|
||||
name: string,
|
||||
description: string,
|
||||
scripts?: {setup: string},
|
||||
command?: {
|
||||
executable?: string,
|
||||
args?: string[]
|
||||
}
|
||||
}
|
||||
|
||||
export interface GoldenConfig extends RendererConfig {
|
||||
file: string;
|
||||
}
|
||||
|
||||
export interface ImageComparisonConfig {
|
||||
rootDirectory: string;
|
||||
analysisThresholds: Array<number>;
|
||||
renderers: Array<RendererConfig>;
|
||||
scenarios: Array<ScenarioConfig>;
|
||||
}
|
||||
|
||||
export interface AnalysisOptions {
|
||||
generateVisuals: boolean;
|
||||
}
|
||||
|
||||
export class ImageComparator {
|
||||
protected imagePixels: number;
|
||||
|
||||
constructor(
|
||||
protected candidateImage: Uint8ClampedArray,
|
||||
protected goldenImage: Uint8ClampedArray,
|
||||
readonly dimensions: Dimensions) {
|
||||
const {width, height} = dimensions;
|
||||
|
||||
this.imagePixels = width * height;
|
||||
}
|
||||
|
||||
protected drawPixel(
|
||||
image: Uint8ClampedArray, position: number, r: number, g: number,
|
||||
b: number, a: number = 255) {
|
||||
image[position + 0] = r;
|
||||
image[position + 1] = g;
|
||||
image[position + 2] = b;
|
||||
image[position + 3] = a;
|
||||
}
|
||||
|
||||
generateVisuals(threshold: number): Visuals {
|
||||
const {candidateImage, goldenImage} = this;
|
||||
const {width, height} = this.dimensions;
|
||||
|
||||
const blackWhiteImage =
|
||||
new Uint8ClampedArray(this.imagePixels * COMPONENTS_PER_PIXEL);
|
||||
const deltaImage =
|
||||
new Uint8ClampedArray(this.imagePixels * COMPONENTS_PER_PIXEL);
|
||||
|
||||
const thresholdSquared = threshold * threshold;
|
||||
|
||||
let maximumDeltaIntensity = 0;
|
||||
|
||||
if (candidateImage.length != goldenImage.length) {
|
||||
throw new Error(`Image sizes do not match (candidate: ${
|
||||
candidateImage.length}, golden: ${goldenImage.length})`);
|
||||
}
|
||||
|
||||
for (let y = 0; y < height; ++y) {
|
||||
for (let x = 0; x < width; ++x) {
|
||||
const index = y * width + x;
|
||||
const position = index * COMPONENTS_PER_PIXEL;
|
||||
const delta =
|
||||
colorDelta(candidateImage, goldenImage, position, position);
|
||||
const exactlyMatched = (delta <= thresholdSquared ? 1 : 0) * 255;
|
||||
|
||||
const thresholdDelta = Math.max(0, delta - thresholdSquared);
|
||||
|
||||
const deltaIntensity =
|
||||
Math.round(255 * thresholdDelta / MAX_COLOR_DISTANCE);
|
||||
|
||||
maximumDeltaIntensity = Math.max(deltaIntensity, maximumDeltaIntensity);
|
||||
|
||||
this.drawPixel(
|
||||
blackWhiteImage,
|
||||
position,
|
||||
exactlyMatched,
|
||||
exactlyMatched,
|
||||
exactlyMatched);
|
||||
this.drawPixel(
|
||||
deltaImage,
|
||||
position,
|
||||
255,
|
||||
255 - deltaIntensity,
|
||||
255 - deltaIntensity);
|
||||
}
|
||||
}
|
||||
|
||||
for (let y = 0; y < height; ++y) {
|
||||
for (let x = 0; x < width; ++x) {
|
||||
const index = y * width + x;
|
||||
const position = index * COMPONENTS_PER_PIXEL;
|
||||
const absoluteDeltaIntensity = 255 - deltaImage![position + 1];
|
||||
const relativeDeltaIntensity = Math.round(
|
||||
255 - 255 * (absoluteDeltaIntensity / maximumDeltaIntensity));
|
||||
|
||||
this.drawPixel(
|
||||
deltaImage,
|
||||
position,
|
||||
255,
|
||||
relativeDeltaIntensity,
|
||||
relativeDeltaIntensity);
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
imageBuffers:
|
||||
{delta: deltaImage.buffer, blackWhite: blackWhiteImage.buffer}
|
||||
};
|
||||
}
|
||||
|
||||
analyze(): ImageComparisonResults {
|
||||
const {candidateImage, goldenImage} = this;
|
||||
const {width, height} = this.dimensions;
|
||||
|
||||
let squareSum = 0;
|
||||
|
||||
if (candidateImage.length != goldenImage.length) {
|
||||
throw new Error(`Image sizes do not match (candidate: ${
|
||||
candidateImage.length}, golden: ${goldenImage.length})`);
|
||||
}
|
||||
|
||||
let modelPixelCount = 0;
|
||||
let colorlessPixelCount = 0;
|
||||
|
||||
for (let y = 0; y < height; ++y) {
|
||||
for (let x = 0; x < width; ++x) {
|
||||
const index = y * width + x;
|
||||
// image's pixel data is stored in an 1-D array, 1st row sequentialy,
|
||||
// then 2nd row, .. for each pixel, its data is stored by order of r, g,
|
||||
// b, a. here position is the index for current pixel's r , position+3
|
||||
// is index for its alpha
|
||||
const position = index * COMPONENTS_PER_PIXEL;
|
||||
// alpha is in range 0~255 here, map it to 0~1
|
||||
const alpha = candidateImage[position + 3] / 255;
|
||||
|
||||
let isWhitePixel = true;
|
||||
let isBlackPixel = true;
|
||||
for (let i = 0; i < 3; i++) {
|
||||
const colorComponent = candidateImage[position + i] * alpha;
|
||||
if (colorComponent != 255) {
|
||||
isWhitePixel = false;
|
||||
}
|
||||
if (colorComponent != 0) {
|
||||
isBlackPixel = false;
|
||||
}
|
||||
}
|
||||
|
||||
if (isBlackPixel || isWhitePixel) {
|
||||
colorlessPixelCount++;
|
||||
}
|
||||
|
||||
if (alpha === 0) {
|
||||
continue;
|
||||
}
|
||||
|
||||
const delta =
|
||||
colorDelta(candidateImage, goldenImage, position, position);
|
||||
|
||||
squareSum += delta * delta;
|
||||
modelPixelCount++;
|
||||
}
|
||||
}
|
||||
|
||||
const imagePixelCount = width * height;
|
||||
|
||||
if (colorlessPixelCount === imagePixelCount) {
|
||||
throw new Error('Candidate image is colorless!');
|
||||
}
|
||||
|
||||
const rmsDistanceRatio =
|
||||
Math.sqrt(squareSum / modelPixelCount) / MAX_COLOR_DISTANCE;
|
||||
return {analysis: {rmsDistanceRatio}};
|
||||
}
|
||||
}
|
||||
|
||||
export function toDecibel(value: number): number {
|
||||
return 10 * Math.log10(value);
|
||||
}
|
||||
@@ -0,0 +1,394 @@
|
||||
/* @license
|
||||
* Copyright 2019 Google LLC. All Rights Reserved.
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
import './images-4-up.js';
|
||||
import '@polymer/paper-slider';
|
||||
|
||||
import {html, LitElement} from 'lit';
|
||||
import {property} from 'lit/decorators.js';
|
||||
|
||||
import {AnalysisCompletedMessage, Dimensions, ImageComparisonMessage, Visuals} from '../common.js';
|
||||
import {ImageAccessor} from '../image-accessor.js';
|
||||
|
||||
|
||||
export class AnalysisView extends LitElement {
|
||||
@property({type: Object}) leftImage: HTMLImageElement|null = null;
|
||||
@property({type: Object}) rightImage: HTMLImageElement|null = null;
|
||||
|
||||
@property({type: Object}) visuals: Visuals|null = null;
|
||||
|
||||
@property({type: Object})
|
||||
protected leftImageAccessor: ImageAccessor|null = null;
|
||||
@property({type: Object})
|
||||
protected rightImageAccessor: ImageAccessor|null = null;
|
||||
@property({type: Object})
|
||||
protected blackWhiteImageAccessor: ImageAccessor|null = null;
|
||||
@property({type: Object})
|
||||
protected deltaImageAccessor: ImageAccessor|null = null;
|
||||
|
||||
@property({type: Number}) protected threshold: number = 5;
|
||||
|
||||
protected worker: Worker;
|
||||
protected port: MessagePort;
|
||||
|
||||
constructor() {
|
||||
super();
|
||||
|
||||
const channel = new MessageChannel();
|
||||
|
||||
this.port = channel.port1;
|
||||
this.worker = new Worker('../dist/image-comparison-worker.js');
|
||||
this.worker.postMessage('connect', [channel.port2]);
|
||||
this.port.onmessage = (event) => this.onMessage(event);
|
||||
this.port.start();
|
||||
}
|
||||
|
||||
onMessage(event: MessageEvent) {
|
||||
const data = event.data as ImageComparisonMessage;
|
||||
const {type} = data;
|
||||
|
||||
switch (type) {
|
||||
case 'analysis-completed':
|
||||
|
||||
this.visuals = (data as AnalysisCompletedMessage).result;
|
||||
|
||||
break;
|
||||
default:
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
async connectedCallback() {
|
||||
super.connectedCallback && super.connectedCallback();
|
||||
|
||||
await this.updateComplete;
|
||||
|
||||
const candidateCanvas =
|
||||
(this.shadowRoot!.querySelector('canvas[slot="top-left"]')! as any)
|
||||
.transferControlToOffscreen();
|
||||
|
||||
const goldenCanvas =
|
||||
(this.shadowRoot!.querySelector('canvas[slot="top-right"]')! as any)
|
||||
.transferControlToOffscreen();
|
||||
const blackWhiteCanvas =
|
||||
(this.shadowRoot!.querySelector('canvas[slot="bottom-left"]')! as any)
|
||||
.transferControlToOffscreen();
|
||||
|
||||
const deltaCanvas =
|
||||
(this.shadowRoot!.querySelector('canvas[slot="bottom-right"]')! as any)
|
||||
.transferControlToOffscreen();
|
||||
|
||||
this.port.postMessage(
|
||||
{
|
||||
type: 'canvases-ready',
|
||||
candidateCanvas,
|
||||
goldenCanvas,
|
||||
blackWhiteCanvas,
|
||||
deltaCanvas
|
||||
},
|
||||
[candidateCanvas, goldenCanvas, blackWhiteCanvas, deltaCanvas]);
|
||||
}
|
||||
|
||||
pick(originatingElement: HTMLImageElement) {
|
||||
if ((originatingElement.parentNode! as HTMLElement)
|
||||
.classList.contains('selected')) {
|
||||
this.deselect(originatingElement);
|
||||
} else {
|
||||
this.select(originatingElement);
|
||||
}
|
||||
}
|
||||
|
||||
get canCompareImages(): boolean {
|
||||
return this.leftImage != null && this.rightImage != null;
|
||||
}
|
||||
|
||||
deselect(element: HTMLImageElement) {
|
||||
if (this.leftImage === element) {
|
||||
this.leftImage = this.rightImage;
|
||||
this.rightImage = null;
|
||||
} else if (this.rightImage === element) {
|
||||
this.rightImage = null;
|
||||
}
|
||||
|
||||
(element.parentNode! as HTMLElement).classList.remove('selected');
|
||||
}
|
||||
|
||||
select(element: HTMLImageElement) {
|
||||
if (this.leftImage == null) {
|
||||
this.leftImage = element;
|
||||
} else if (this.rightImage == null) {
|
||||
this.rightImage = element;
|
||||
} else {
|
||||
return;
|
||||
}
|
||||
|
||||
(element.parentNode! as HTMLElement).classList.add('selected');
|
||||
}
|
||||
|
||||
enterAnalysisView() {
|
||||
if (this.canCompareImages) {
|
||||
this.classList.add('compare');
|
||||
}
|
||||
}
|
||||
|
||||
exitAnalysisView() {
|
||||
this.classList.remove('compare');
|
||||
}
|
||||
|
||||
reset() {
|
||||
if (this.rightImage != null) {
|
||||
this.deselect(this.rightImage);
|
||||
}
|
||||
|
||||
if (this.leftImage != null) {
|
||||
this.deselect(this.leftImage);
|
||||
}
|
||||
}
|
||||
|
||||
get comparisonDimensions(): Dimensions {
|
||||
if (this.canCompareImages) {
|
||||
const width =
|
||||
Math.max(this.leftImage!.naturalWidth, this.rightImage!.naturalWidth);
|
||||
const height = Math.max(
|
||||
this.leftImage!.naturalHeight, this.rightImage!.naturalHeight);
|
||||
|
||||
return {width, height};
|
||||
}
|
||||
|
||||
return {width: 0, height: 0};
|
||||
}
|
||||
|
||||
async updated(changedProperties: Map<string, any>) {
|
||||
super.updated(changedProperties);
|
||||
|
||||
if (changedProperties.has('analysisResult') && this.visuals != null &&
|
||||
this.canCompareImages) {
|
||||
const {imageBuffers} = this.visuals;
|
||||
const {width, height} = this.comparisonDimensions;
|
||||
|
||||
this.blackWhiteImageAccessor = ImageAccessor.fromArrayBuffer(
|
||||
imageBuffers.blackWhite!, width, height);
|
||||
|
||||
this.deltaImageAccessor =
|
||||
ImageAccessor.fromArrayBuffer(imageBuffers.delta!, width, height);
|
||||
}
|
||||
|
||||
if ((changedProperties.has('leftImage') ||
|
||||
changedProperties.has('rightImage')) &&
|
||||
this.canCompareImages) {
|
||||
this.leftImageAccessor = ImageAccessor.fromImageElement(this.leftImage!);
|
||||
this.rightImageAccessor =
|
||||
ImageAccessor.fromImageElement(this.rightImage!);
|
||||
|
||||
const {width, height} = this.comparisonDimensions;
|
||||
const candidateImageBuffer = this.leftImageAccessor.toArrayBuffer();
|
||||
const goldenImageBuffer = this.rightImageAccessor.toArrayBuffer();
|
||||
|
||||
this.port.postMessage(
|
||||
{
|
||||
type: 'images-assigned',
|
||||
candidateImageBuffer,
|
||||
goldenImageBuffer,
|
||||
dimensions: {width, height}
|
||||
},
|
||||
[
|
||||
candidateImageBuffer as ArrayBuffer,
|
||||
goldenImageBuffer as ArrayBuffer
|
||||
]);
|
||||
|
||||
this.port.postMessage(
|
||||
{type: 'threshold-changed', threshold: this.threshold});
|
||||
}
|
||||
|
||||
if (changedProperties.has('threshold') && this.canCompareImages) {
|
||||
this.port.postMessage(
|
||||
{type: 'threshold-changed', threshold: this.threshold});
|
||||
}
|
||||
}
|
||||
|
||||
render() {
|
||||
let instructions;
|
||||
|
||||
if (this.canCompareImages) {
|
||||
instructions = html`
|
||||
<button @click="${() => this.enterAnalysisView()}">
|
||||
Compare selected images
|
||||
</button>
|
||||
<button @click="${() => this.reset()}" class="no-border">
|
||||
Clear selections
|
||||
</button>`;
|
||||
} else {
|
||||
instructions = html`<p>Select two images to compare them!</p>`;
|
||||
}
|
||||
|
||||
return html`
|
||||
<style>
|
||||
:host {
|
||||
display: block;
|
||||
position: fixed;
|
||||
width: 100vw;
|
||||
bottom: 0;
|
||||
left: 0;
|
||||
|
||||
font-family: Google Sans, sans-serif;
|
||||
background-color: #37474f;
|
||||
color: #fff;
|
||||
|
||||
box-shadow: 0px 3px 12px rgba(100, 100, 100, 0.3);
|
||||
}
|
||||
|
||||
#instructions {
|
||||
text-align: center;
|
||||
font-style: italic;
|
||||
padding: 1em;
|
||||
}
|
||||
|
||||
button {
|
||||
font-family: Google Sans, sans-serif;
|
||||
text-transform: uppercase;
|
||||
color: #fff;
|
||||
background-color: #37474f;
|
||||
border: 1px solid #fff;
|
||||
border-radius: 3px;
|
||||
font-size: 1.1em;
|
||||
font-weight: 200;
|
||||
padding: 0.45em 0.85em;
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
button {
|
||||
margin: 0.35em 0;
|
||||
}
|
||||
|
||||
button:hover, button:focus {
|
||||
background-color: #62727b;
|
||||
}
|
||||
|
||||
#comparison {
|
||||
display: flex;
|
||||
position: fixed;
|
||||
flex-direction: column;
|
||||
align-items: stretch;
|
||||
justify-content: center;
|
||||
top: 0;
|
||||
left: 0;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
box-sizing: border-box;
|
||||
background-color: #37474f;
|
||||
transform: translateY(1vh);
|
||||
opacity: 0;
|
||||
pointer-events: none;
|
||||
transition: transform 0.3s, opacity 0.3s;
|
||||
}
|
||||
|
||||
:host(.compare) #comparison {
|
||||
transform: translateY(0);
|
||||
opacity: 1;
|
||||
pointer-events: unset;
|
||||
}
|
||||
|
||||
button.no-border {
|
||||
border: none;
|
||||
}
|
||||
|
||||
.exit {
|
||||
margin: 0;
|
||||
border: 0;
|
||||
font-size: 0;
|
||||
line-height: 0;
|
||||
|
||||
position: absolute;
|
||||
right: 32px;
|
||||
top: 32px;
|
||||
width: 32px;
|
||||
height: 32px;
|
||||
opacity: 0.75;
|
||||
}
|
||||
|
||||
.exit:hover {
|
||||
opacity: 1;
|
||||
}
|
||||
|
||||
.exit:before, .exit:after {
|
||||
content: ' ';
|
||||
position: absolute;
|
||||
top: 0px;
|
||||
left: 15px;
|
||||
height: 33px;
|
||||
width: 2px;
|
||||
background-color: #fff;
|
||||
}
|
||||
|
||||
.exit:before {
|
||||
transform: rotate(45deg);
|
||||
}
|
||||
.exit:after {
|
||||
transform: rotate(-45deg);
|
||||
}
|
||||
|
||||
images-4-up {
|
||||
flex: 1 1 100%;
|
||||
}
|
||||
|
||||
#controls {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
flex: 1 1 90px;
|
||||
|
||||
color: #fff;
|
||||
--primary-text-color: #fff;
|
||||
--paper-slider-active-color: #fff;
|
||||
--paper-slider-pin-color: #fff;
|
||||
--paper-slider-knob-color: #fff;
|
||||
--paper-slider-font-color: #444;
|
||||
}
|
||||
|
||||
canvas {
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
}
|
||||
</style>
|
||||
<div id="instructions">
|
||||
${instructions}
|
||||
</div>
|
||||
<div id="comparison">
|
||||
<images-4-up .dimensions="${this.comparisonDimensions}"
|
||||
.topLeftImageAccessor="${this.leftImageAccessor}"
|
||||
.topRightImageAccessor="${this.rightImageAccessor}"
|
||||
.bottomLeftImageAccessor="${this.blackWhiteImageAccessor}"
|
||||
.bottomRightImageAccessor="${this.deltaImageAccessor}">
|
||||
<canvas slot="top-left"></canvas>
|
||||
<canvas slot="top-right"></canvas>
|
||||
<canvas slot="bottom-left"></canvas>
|
||||
<canvas slot="bottom-right"></canvas>
|
||||
</images-4-up>
|
||||
<div id="controls">
|
||||
<label id="threshold-label">Threshold</label>
|
||||
<paper-slider
|
||||
@change="${
|
||||
(event: Event) => this.threshold =
|
||||
parseInt((event.target as HTMLInputElement).value, 10)}"
|
||||
aria-labelledby="threshold-label"
|
||||
min="0" max="10" step="1" pin snaps value="5" editable>
|
||||
</paper-slider>
|
||||
</div>
|
||||
<button class="exit" @click="${() => this.exitAnalysisView()}">Exit</button>
|
||||
</div>`
|
||||
}
|
||||
}
|
||||
|
||||
customElements.define('analysis-view', AnalysisView);
|
||||
@@ -0,0 +1,108 @@
|
||||
/* @license
|
||||
* Copyright 2019 Google LLC. All Rights Reserved.
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
import './analysis-view.js';
|
||||
import './rendering-scenario.js';
|
||||
|
||||
import {html, LitElement} from 'lit';
|
||||
import {property, query} from 'lit/decorators.js';
|
||||
|
||||
import {ImageComparisonConfig} from '../common.js';
|
||||
|
||||
import {AnalysisView} from './analysis-view.js';
|
||||
|
||||
export class ImageComparisonApp extends LitElement {
|
||||
@property({type: String}) src: string = '';
|
||||
|
||||
@property({type: Object}) config: ImageComparisonConfig|null = null;
|
||||
|
||||
@query('analysis-view') analysisView!: AnalysisView;
|
||||
|
||||
updated(changedProperties: Map<any, any>) {
|
||||
super.updated(changedProperties);
|
||||
|
||||
if (changedProperties.has('src')) {
|
||||
this.loadConfig();
|
||||
}
|
||||
}
|
||||
|
||||
connectedCallback() {
|
||||
super.connectedCallback && super.connectedCallback();
|
||||
window.addEventListener('hashchange', this.onHashChange);
|
||||
}
|
||||
|
||||
disconnectedCallback() {
|
||||
super.connectedCallback && super.connectedCallback();
|
||||
window.removeEventListener('hashchange', this.onHashChange);
|
||||
}
|
||||
|
||||
private onHashChange = () => {
|
||||
if (!location.hash) {
|
||||
return;
|
||||
}
|
||||
const targetElement = this.shadowRoot!.querySelector(`${location.hash}`);
|
||||
if (targetElement) {
|
||||
targetElement.scrollIntoView();
|
||||
}
|
||||
};
|
||||
|
||||
private async loadConfig() {
|
||||
if (this.src) {
|
||||
this.config = await (await fetch(this.src)).json();
|
||||
} else {
|
||||
this.config = null;
|
||||
}
|
||||
await this.updateComplete;
|
||||
this.onHashChange();
|
||||
}
|
||||
|
||||
private selectElement(event: {detail: {element: HTMLImageElement;};}) {
|
||||
this.analysisView.pick(event.detail.element);
|
||||
}
|
||||
|
||||
render() {
|
||||
const {config} = this;
|
||||
|
||||
if (config == null) {
|
||||
return this.src ? html`Loading...` : html`No config specified`;
|
||||
}
|
||||
|
||||
const goldens = config.renderers.map(
|
||||
renderer => ({...renderer, file: `${renderer.name}-golden.png`}));
|
||||
|
||||
const scenarios = config!.scenarios.map((scenario) => html`
|
||||
<rendering-scenario id="${scenario.name}"
|
||||
.name="${scenario.name}"
|
||||
.goldens="${goldens}"
|
||||
.dimensions="${scenario.dimensions}"
|
||||
.exclude="${scenario.exclude || []}"
|
||||
@select="${this.selectElement}">
|
||||
</rendering-scenario>`);
|
||||
|
||||
return html`
|
||||
<style>
|
||||
#scenarios {
|
||||
max-width: 100%;
|
||||
padding-bottom: 5em;
|
||||
}
|
||||
</style>
|
||||
<div id="scenarios">
|
||||
${scenarios}
|
||||
</div>
|
||||
<analysis-view></analysis-view>`;
|
||||
}
|
||||
};
|
||||
|
||||
customElements.define('image-comparison-app', ImageComparisonApp);
|
||||
@@ -0,0 +1,175 @@
|
||||
/* @license
|
||||
* Copyright 2019 Google LLC. All Rights Reserved.
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
import '../image-accessor.js';
|
||||
import './magnifying-glass.js';
|
||||
|
||||
import {html, LitElement} from 'lit';
|
||||
import {property} from 'lit/decorators.js';
|
||||
|
||||
import {Dimensions, Pixel, Rect} from '../common.js';
|
||||
import {ImageAccessor} from '../image-accessor.js';
|
||||
|
||||
|
||||
export class Images4Up extends LitElement {
|
||||
@property({type: Object}) topLeftImageAccessor: ImageAccessor|null = null;
|
||||
@property({type: Object}) topRightImageAccessor: ImageAccessor|null = null;
|
||||
@property({type: Object}) bottomLeftImageAccessor: ImageAccessor|null = null;
|
||||
@property({type: Object}) bottomRightImageAccessor: ImageAccessor|null = null;
|
||||
|
||||
@property({type: Object}) magnifiedPixel: Pixel|null = null;
|
||||
@property({type: Object}) dimensions: Dimensions = {width: 0, height: 0};
|
||||
|
||||
@property({type: Object})
|
||||
protected containerRect: Rect = {x: 0, y: 0, width: 0, height: 0};
|
||||
|
||||
localCoordinateToImagePixel(localX: number, localY: number): Pixel {
|
||||
const constrainedX = Math.min(
|
||||
Math.max(localX - this.containerRect.x, -1),
|
||||
this.containerRect.width + 1);
|
||||
const constrainedY = Math.min(
|
||||
Math.max(localY - this.containerRect.y, -1),
|
||||
this.containerRect.height + 1);
|
||||
|
||||
const scaledWidth = this.containerRect.width / 2;
|
||||
const scaledHeight = this.containerRect.height / 2;
|
||||
|
||||
const scale = this.dimensions.width / scaledWidth;
|
||||
|
||||
return {
|
||||
x: Math.floor((constrainedX % scaledWidth) * scale),
|
||||
y: Math.floor((constrainedY % scaledHeight) * scale)
|
||||
};
|
||||
}
|
||||
|
||||
protected selectPixel(pixel: Pixel) {
|
||||
this.magnifiedPixel = pixel;
|
||||
this.dispatchEvent(new CustomEvent('select-pixel', {detail: pixel}));
|
||||
}
|
||||
|
||||
onPointerEvent(event: PointerEvent) {
|
||||
const pixel = this.localCoordinateToImagePixel(event.x, event.y);
|
||||
|
||||
if (pixel.x > -1 && pixel.x < this.dimensions.width && pixel.y > -1 &&
|
||||
pixel.y < this.dimensions.height) {
|
||||
this.selectPixel(pixel);
|
||||
} else {
|
||||
this.magnifiedPixel = null;
|
||||
}
|
||||
}
|
||||
|
||||
connectedCallback() {
|
||||
super.connectedCallback && super.connectedCallback();
|
||||
self.addEventListener('resize', (_event) => this.updateSize());
|
||||
}
|
||||
|
||||
updateSize() {
|
||||
const rect = this.getBoundingClientRect();
|
||||
const {width: containerWidth, height: containerHeight} = rect;
|
||||
const fourUpWidth = 2 * this.dimensions.width;
|
||||
const fourUpHeight = 2 * this.dimensions.height;
|
||||
|
||||
const containerAspectRatio = containerWidth / containerHeight;
|
||||
const fourUpAspectRatio = fourUpWidth / fourUpHeight;
|
||||
|
||||
const scale = fourUpAspectRatio < containerAspectRatio ?
|
||||
containerHeight / fourUpHeight :
|
||||
containerWidth / fourUpWidth;
|
||||
|
||||
const width = scale * fourUpWidth;
|
||||
const height = scale * fourUpHeight;
|
||||
|
||||
const x = (containerWidth - width) / 2;
|
||||
const y = (containerHeight - height) / 2;
|
||||
|
||||
this.containerRect = {x, y, width, height};
|
||||
}
|
||||
|
||||
async updated(changedProperties: Map<string, any>) {
|
||||
super.updated(changedProperties);
|
||||
|
||||
if (changedProperties.has('dimensions') && this.dimensions != null) {
|
||||
this.updateSize();
|
||||
|
||||
this.selectPixel({
|
||||
x: Math.floor(this.dimensions.width / 2),
|
||||
y: Math.floor(this.dimensions.height / 2)
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
render() {
|
||||
const {x, y, width, height} = this.containerRect;
|
||||
|
||||
return html`
|
||||
<style>
|
||||
:host {
|
||||
display: block;
|
||||
position: relative;
|
||||
}
|
||||
|
||||
magnifying-glass {
|
||||
flex: 1 1 50%;
|
||||
width: 50%;
|
||||
height: 50%;
|
||||
}
|
||||
|
||||
#container {
|
||||
display: flex;
|
||||
cursor: crosshair;
|
||||
position: absolute;
|
||||
flex-direction: row;
|
||||
flex-wrap: wrap;
|
||||
align-items: flex-start;
|
||||
justify-content: flex-start;
|
||||
}
|
||||
|
||||
#magnifier-button {
|
||||
position: absolute;
|
||||
top: 0;
|
||||
right: 0;
|
||||
}
|
||||
</style>
|
||||
<div id="container"
|
||||
@pointermove="${(event: PointerEvent) => this.onPointerEvent(event)}"
|
||||
@pointerdown="${(event: PointerEvent) => this.onPointerEvent(event)}"
|
||||
style="transform: translate(${x}px, ${y}px);
|
||||
width: ${width}px;
|
||||
height: ${height}px">
|
||||
<magnifying-glass id="top-left"
|
||||
.pixel="${this.magnifiedPixel}"
|
||||
.imageAccessor="${this.topLeftImageAccessor}">
|
||||
<slot name="top-left"></slot>
|
||||
</magnifying-glass>
|
||||
<magnifying-glass id="top-right"
|
||||
.pixel="${this.magnifiedPixel}"
|
||||
.imageAccessor="${this.topRightImageAccessor}">
|
||||
<slot name="top-right"></slot>
|
||||
</magnifying-glass>
|
||||
<magnifying-glass id="bottom-left"
|
||||
.pixel="${this.magnifiedPixel}"
|
||||
.imageAccessor="${this.bottomLeftImageAccessor}">
|
||||
<slot name="bottom-left"></slot>
|
||||
</magnifying-glass>
|
||||
<magnifying-glass id="bottom-right"
|
||||
.pixel="${this.magnifiedPixel}"
|
||||
.imageAccessor="${this.bottomRightImageAccessor}">
|
||||
<slot name="bottom-right"></slot>
|
||||
</magnifying-glass>
|
||||
</div>`;
|
||||
}
|
||||
}
|
||||
|
||||
customElements.define('images-4-up', Images4Up);
|
||||
@@ -0,0 +1,185 @@
|
||||
/* @license
|
||||
* Copyright 2019 Google LLC. All Rights Reserved.
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
|
||||
import {html, LitElement} from 'lit';
|
||||
import {property} from 'lit/decorators.js';
|
||||
|
||||
import {Pixel} from '../common.js';
|
||||
import {ImageAccessor} from '../image-accessor.js';
|
||||
|
||||
const PIXEL_SIZE = 10;
|
||||
|
||||
export class MagnifyingGlass extends LitElement {
|
||||
@property({type: Object}) imageAccessor: ImageAccessor|null = null;
|
||||
@property({type: Object}) pixel: Pixel|null = null;
|
||||
@property({type: String}) direction: string = 'vertical';
|
||||
|
||||
@property({type: Object}) position: Pixel = {x: 0, y: 0};
|
||||
|
||||
protected context: CanvasRenderingContext2D|null = null;
|
||||
|
||||
@property({type: Boolean}) protected xRay: boolean = true;
|
||||
@property({type: Number}) protected reticleSize: number = 1;
|
||||
|
||||
protected enhance() {
|
||||
if (this.pixel == null || this.imageAccessor == null) {
|
||||
return;
|
||||
}
|
||||
|
||||
this.classList.remove('hidden');
|
||||
|
||||
if (this.context == null) {
|
||||
const canvas = this.shadowRoot!.querySelector('canvas');
|
||||
|
||||
if (canvas == null) {
|
||||
return;
|
||||
}
|
||||
|
||||
this.context = canvas.getContext('2d')!;
|
||||
}
|
||||
|
||||
const {context} = this;
|
||||
const rect = this.getBoundingClientRect();
|
||||
const scale = rect.width / this.imageAccessor!.width;
|
||||
|
||||
for (let y = 0; y < 24; ++y) {
|
||||
for (let x = 0; x < 24; ++x) {
|
||||
context.fillStyle = this.imageAccessor!.cssColorAt(
|
||||
this.pixel.x + x - 12, this.pixel.y + y - 12);
|
||||
context.fillRect(
|
||||
x * PIXEL_SIZE, y * PIXEL_SIZE, PIXEL_SIZE, PIXEL_SIZE);
|
||||
}
|
||||
}
|
||||
|
||||
this.reticleSize = Math.round(26 * scale);
|
||||
this.position = {
|
||||
x: this.pixel.x * scale - 130,
|
||||
y: this.pixel.y * scale - 130
|
||||
};
|
||||
}
|
||||
|
||||
toggleXRay() {
|
||||
this.xRay = !this.xRay;
|
||||
}
|
||||
|
||||
protected hide() {
|
||||
this.classList.add('hidden');
|
||||
}
|
||||
|
||||
get glassPosition() {
|
||||
const {width, height} =
|
||||
this.imageAccessor != null ? this.imageAccessor : {width: 0, height: 0};
|
||||
const {x, y} = this.pixel != null ? this.pixel : {x: 0, y: 0};
|
||||
|
||||
return this.direction === 'horizontal' ?
|
||||
x < (width / 2) ? 'right' : 'left' :
|
||||
y < (height / 2) ? 'bottom' : 'top';
|
||||
}
|
||||
|
||||
render() {
|
||||
this.enhance();
|
||||
|
||||
return html`
|
||||
<style>
|
||||
:host {
|
||||
display: block;
|
||||
position: relative;
|
||||
}
|
||||
|
||||
#glass {
|
||||
display: block;
|
||||
position: absolute;
|
||||
box-sizing: border-box;
|
||||
top: 0;
|
||||
left: 0;
|
||||
width: 260px;
|
||||
height: 260px;
|
||||
transition: transform 0.1s;
|
||||
z-index: 1;
|
||||
background-color: transparent;
|
||||
}
|
||||
|
||||
canvas {
|
||||
box-sizing: border-box;
|
||||
box-shadow: 0 2px 10px rgba(40, 40, 40, 0.25);
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
transition: opacity 0.3s, transform 0.5s;
|
||||
opacity: 1;
|
||||
}
|
||||
|
||||
#glass.top canvas {
|
||||
transform: translate(0, -180px);
|
||||
}
|
||||
|
||||
#glass.bottom canvas {
|
||||
transform: translate(0, 180px);
|
||||
}
|
||||
|
||||
#glass.left canvas {
|
||||
transform: translate(-180px, 0);
|
||||
}
|
||||
|
||||
#glass.right canvas {
|
||||
transform: translate(180px, 0);
|
||||
}
|
||||
|
||||
#glass:before {
|
||||
content: '';
|
||||
display: block;
|
||||
position: absolute;
|
||||
top: 0;
|
||||
left: 0;
|
||||
transform: translate(calc(-50% + 130px), calc(130px - var(--reticle-size) / 2));
|
||||
box-sizing: border-box;
|
||||
border: 1px solid black;
|
||||
width: var(--reticle-size);
|
||||
height: var(--reticle-size);
|
||||
}
|
||||
|
||||
#glass:after {
|
||||
content: '';
|
||||
display: block;
|
||||
position: absolute;
|
||||
top: 0;
|
||||
left: 0;
|
||||
transform: translate(calc(-50% + 130px), calc(129px - var(--reticle-size) / 2));
|
||||
box-sizing: border-box;
|
||||
|
||||
border: 1px solid white;
|
||||
|
||||
width: calc(var(--reticle-size) + 2px);
|
||||
height: calc(var(--reticle-size) + 2px);
|
||||
}
|
||||
|
||||
:host(.hidden) canvas {
|
||||
opacity: 0;
|
||||
}
|
||||
</style>
|
||||
<slot></slot>
|
||||
<div id="glass"
|
||||
class="${this.glassPosition}"
|
||||
style="transform: translate(${this.position.x}px, ${
|
||||
this.position.y}px); --reticle-size: ${this.reticleSize}px"
|
||||
@click="${() => this.toggleXRay()}">
|
||||
<canvas style="opacity: ${this.xRay ? 0 : 1};" width="240" height="240">
|
||||
</div>
|
||||
</canvas>
|
||||
`;
|
||||
}
|
||||
}
|
||||
|
||||
customElements.define('magnifying-glass', MagnifyingGlass);
|
||||
@@ -0,0 +1,153 @@
|
||||
/* @license
|
||||
* Copyright 2019 Google LLC. All Rights Reserved.
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
import {css, html, LitElement} from 'lit';
|
||||
import {customElement, property} from 'lit/decorators.js'
|
||||
|
||||
import {ImageComparisonConfig, ScenarioConfig} from '../common.js';
|
||||
import {ConfigReader} from '../config-reader.js';
|
||||
|
||||
@customElement('renderer-harness')
|
||||
export class RendererConfiguration extends LitElement {
|
||||
@property({type: String, attribute: 'scenario-name'})
|
||||
scenarioName: string|null = null;
|
||||
|
||||
@property({type: String, attribute: 'config-url'})
|
||||
configUrl: string|null = null;
|
||||
|
||||
@property({type: Boolean, attribute: 'hide-ui'}) hideUi: boolean = false;
|
||||
|
||||
@property({type: Object}) protected config: ImageComparisonConfig|null = null;
|
||||
|
||||
@property({type: Object}) protected scenario: ScenarioConfig|null = null;
|
||||
|
||||
connectedCallback() {
|
||||
super.connectedCallback();
|
||||
|
||||
const {queryParameters} = this;
|
||||
this.configUrl = queryParameters.config || '../../config.json';
|
||||
this.scenarioName = queryParameters.scenario || null;
|
||||
this.hideUi = 'hide-ui' in queryParameters || false;
|
||||
}
|
||||
|
||||
protected get queryParameters(): {[index: string]: string} {
|
||||
return self.location.search.slice(1).split('&').reduce(
|
||||
(queryParameters, parameter) => {
|
||||
const [key, value] = parameter.split('=');
|
||||
queryParameters[key] = decodeURIComponent(value);
|
||||
return queryParameters;
|
||||
},
|
||||
{} as {[index: string]: string});
|
||||
}
|
||||
|
||||
async updated(changedProperties: Map<string, any>) {
|
||||
super.updated(changedProperties);
|
||||
|
||||
if (changedProperties.has('configUrl')) {
|
||||
if (this.configUrl == null) {
|
||||
this.config = null;
|
||||
} else {
|
||||
this.config = await (await fetch(this.configUrl)).json();
|
||||
}
|
||||
}
|
||||
|
||||
if (changedProperties.has('scenarioName') ||
|
||||
changedProperties.has('configUrl')) {
|
||||
if (this.scenarioName == null || this.config == null) {
|
||||
this.scenario = null;
|
||||
} else {
|
||||
this.scenario =
|
||||
(new ConfigReader(this.config)).scenarioConfig(this.scenarioName);
|
||||
}
|
||||
|
||||
const previousScenarioName = changedProperties.get('scenarioName');
|
||||
|
||||
if (previousScenarioName !== this.scenarioName) {
|
||||
this.dispatchEvent(new CustomEvent(
|
||||
'scenario-change', {detail: {scenario: this.scenario}}));
|
||||
}
|
||||
|
||||
if (previousScenarioName != null && this.scenarioName != null) {
|
||||
const url = new URL(window.location.toString());
|
||||
url.search = url.search.replace(
|
||||
`scenario=${encodeURIComponent(previousScenarioName)}`,
|
||||
`scenario=${encodeURIComponent(this.scenarioName)}`);
|
||||
history.pushState(null, document.title, url.toString());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
static get styles() {
|
||||
return css`
|
||||
:host {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
font-family: system-ui, sans-serif;
|
||||
color: #fafafa;
|
||||
}
|
||||
|
||||
#widgets {
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
background-color: #444;
|
||||
padding: 0 1em;
|
||||
min-height: 4em;
|
||||
}
|
||||
|
||||
#widgets > * {
|
||||
margin-right: 1em;
|
||||
}
|
||||
|
||||
#widgets.hidden {
|
||||
display: none;
|
||||
}
|
||||
|
||||
#renderer {
|
||||
max-width: 100%;
|
||||
overflow: auto;
|
||||
}
|
||||
`;
|
||||
}
|
||||
|
||||
render() {
|
||||
const widgets = [];
|
||||
|
||||
if (this.config != null) {
|
||||
const scenarioOptions = this.config.scenarios.map(
|
||||
scenario => html`<option ?selected="${
|
||||
scenario.name === this.scenarioName}">${scenario.name}</option>`);
|
||||
widgets.push(html`
|
||||
<select @change="${
|
||||
(event: Event) => this.scenarioName =
|
||||
(event.target != null ?
|
||||
(event.target as HTMLSelectElement).value :
|
||||
this.scenarioName)}">
|
||||
${scenarioOptions}
|
||||
</select>`);
|
||||
}
|
||||
|
||||
return html`
|
||||
<div id="widgets" class="${this.hideUi ? 'hidden' : ''}">
|
||||
<h2><slot name="title"></slot></h2>
|
||||
${widgets}
|
||||
</div>
|
||||
<div id="renderer" .style="${this.hideUi ? 'overflow: visible' : ''}">
|
||||
<slot name="renderer"></slot>
|
||||
</div>
|
||||
`;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,262 @@
|
||||
/* @license
|
||||
* Copyright 2019 Google LLC. All Rights Reserved.
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
import {html, LitElement} from 'lit';
|
||||
import {property} from 'lit/decorators.js';
|
||||
|
||||
import {Dimensions, GoldenConfig, ImageComparisonAnalysis, ScenarioRecord, toDecibel} from '../common.js';
|
||||
|
||||
|
||||
const DEFAULT_DIMENSIONS: Dimensions = {
|
||||
width: 0,
|
||||
height: 0
|
||||
};
|
||||
|
||||
export class RenderingScenario extends LitElement {
|
||||
@property({type: String}) name: string = '';
|
||||
|
||||
@property({type: Object}) goldens: Array<GoldenConfig> = [];
|
||||
|
||||
@property({type: Object}) dimensions: Dimensions = DEFAULT_DIMENSIONS;
|
||||
|
||||
@property({type: Array}) exclude: Array<string> = [];
|
||||
|
||||
@property({type: Object}) analysis: ScenarioRecord|null = null;
|
||||
|
||||
get basePath() {
|
||||
if (!this.name) {
|
||||
return '';
|
||||
}
|
||||
|
||||
return `./results/${this.name}`;
|
||||
}
|
||||
|
||||
async connectedCallback() {
|
||||
super.connectedCallback && super.connectedCallback();
|
||||
|
||||
this.shadowRoot!.addEventListener('click', (event) => {
|
||||
const originatingElement = event.target as HTMLImageElement;
|
||||
|
||||
if (originatingElement == null || originatingElement.tagName !== 'IMG') {
|
||||
return;
|
||||
}
|
||||
|
||||
this.dispatchEvent(
|
||||
new CustomEvent('select', {detail: {element: originatingElement}}));
|
||||
});
|
||||
}
|
||||
|
||||
updated(changedProperties: Map<any, any>) {
|
||||
super.updated(changedProperties);
|
||||
|
||||
if (this.analysis == null) {
|
||||
this.loadAnalysis();
|
||||
}
|
||||
}
|
||||
|
||||
private async loadAnalysis() {
|
||||
const analysisPath = `${this.basePath}/analysis.json`;
|
||||
this.analysis = await (await fetch(analysisPath)).json();
|
||||
}
|
||||
|
||||
private metricTemplate(
|
||||
analysisResults: ImageComparisonAnalysis, goldenName: string) {
|
||||
const rmsInDecibel = toDecibel(analysisResults.rmsDistanceRatio).toFixed(2);
|
||||
return html` <span>${rmsInDecibel} dB </span>
|
||||
<div class="tooltip">
|
||||
<span class="question-icon"> </span>
|
||||
<span class="tooltiptext">
|
||||
Root mean square(RMS) color distance between ${goldenName}'s golden
|
||||
and model-viewer's on rendering ${this.name} in decibels.
|
||||
The decibel is given by: 10 * log(RMS). More negative means a closer match.
|
||||
</span>
|
||||
</div>`
|
||||
}
|
||||
|
||||
render() {
|
||||
const {basePath} = this;
|
||||
const {width} = this.dimensions;
|
||||
|
||||
const images =
|
||||
this.goldens.filter(golden => !this.exclude.includes(golden.name))
|
||||
.map(
|
||||
(golden, index) => html`
|
||||
<div class="screenshot">
|
||||
<header>
|
||||
<h2>${golden.description}</h2>
|
||||
</header>
|
||||
<div class="check"></div>
|
||||
<img data-id="${this.name} ${golden.name}"
|
||||
style="width:${width}px" src="${basePath}/${golden.file}">
|
||||
<div class = "metrics">
|
||||
${
|
||||
index == 0 || this.analysis == null ?
|
||||
html`<span>---</span>` :
|
||||
this.metricTemplate(
|
||||
this.analysis.analysisResults[index - 1],
|
||||
golden.name)}
|
||||
</div>
|
||||
</div>`);
|
||||
|
||||
return html`
|
||||
|
||||
<style>
|
||||
:host {
|
||||
display: inline-flex;
|
||||
flex-direction: column;
|
||||
flex: 0 1 auto;
|
||||
position: relative;
|
||||
margin: 1em;
|
||||
padding: 1em;
|
||||
border-radius: 3px;
|
||||
box-shadow: 0px 2px 8px rgba(100, 100, 100, 0.2);
|
||||
background-color: #fff;
|
||||
color: #555;
|
||||
font-family: Google Sans, sans-serif;
|
||||
}
|
||||
|
||||
h1 {
|
||||
font-size: 1.5em;
|
||||
margin: 0.166em 0.33em 0.33em;
|
||||
}
|
||||
|
||||
.tooltip {
|
||||
position: relative;
|
||||
display: inline-block;
|
||||
}
|
||||
|
||||
.tooltip .tooltiptext {
|
||||
opacity: 0.4;
|
||||
visibility: hidden;
|
||||
width: 200px;
|
||||
font-size: 0.8em;
|
||||
background-color: black;
|
||||
color: #fff;
|
||||
text-align: left;
|
||||
border-radius: 6px;
|
||||
padding: 5px 5px;
|
||||
position: absolute;
|
||||
z-index: 1;
|
||||
}
|
||||
|
||||
.tooltip:hover .tooltiptext {
|
||||
visibility: visible;
|
||||
}
|
||||
|
||||
.question-icon {
|
||||
background-image: url(../../shared-assets/icons/help_outline-24px.svg);
|
||||
background-size : 0.8em;
|
||||
background-repeat: no-repeat;
|
||||
background-position: 0% 100%;
|
||||
width: 1em;
|
||||
height: 1em;
|
||||
display: inline-block;
|
||||
}
|
||||
|
||||
.metrics {
|
||||
text-align: center;
|
||||
height: 2em;
|
||||
}
|
||||
|
||||
.screenshot > header {
|
||||
display: block;
|
||||
position: absolute;
|
||||
width: 100%;
|
||||
bottom: 2em;
|
||||
left: 0;
|
||||
text-align: center;
|
||||
pointer-events: none;
|
||||
}
|
||||
|
||||
h2 {
|
||||
font-size: 1em;
|
||||
|
||||
display: inline-block;
|
||||
box-sizing: border-box;
|
||||
|
||||
border-radius: 3px;
|
||||
padding: 0.25em 0.5em;
|
||||
margin: 0;
|
||||
|
||||
background-color: rgba(255, 255, 255, 0.75);
|
||||
pointer-events: none;
|
||||
}
|
||||
|
||||
#screenshots {
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
width: 100%;
|
||||
height: 544px;
|
||||
align-items: center;
|
||||
flex-wrap: nowrap;
|
||||
}
|
||||
|
||||
.screenshot {
|
||||
box-sizing: border-box;
|
||||
display: flex;
|
||||
position: relative;
|
||||
flex-direction: column;
|
||||
object-fit: contain;
|
||||
margin: 0.5em;
|
||||
transition: transform 0.3s;
|
||||
}
|
||||
|
||||
.screenshot > img {
|
||||
max-width: 256px;
|
||||
flex: 0 0 0;
|
||||
}
|
||||
|
||||
.check {
|
||||
display: flex;
|
||||
position: absolute;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
box-sizing: bborder-box;
|
||||
top: 0.5em;;
|
||||
right: 0.5em;
|
||||
width: 1em;
|
||||
height: 1em;
|
||||
border-radius: 2em;
|
||||
border: 2px solid #fff;
|
||||
background-color: #f8f8f8;
|
||||
}
|
||||
|
||||
.selected .check {
|
||||
background-color: #37474f;
|
||||
}
|
||||
|
||||
.selected .check:after {
|
||||
content: '';
|
||||
display: block;
|
||||
width: 0.25em;
|
||||
height: 0.5em;
|
||||
border: solid #fff;
|
||||
border-width: 0 2px 2px 0;
|
||||
transform: translate(-0em, -0.1em) rotate(45deg);
|
||||
}
|
||||
|
||||
.selected {
|
||||
transform: translateY(-0.5em) scale(1.025);
|
||||
box-shadow: 0px 6px 12px rgba(100, 100, 100, 0.2);
|
||||
}
|
||||
</style>
|
||||
<h1><a href="#${this.name}">${this.name}</a></h1>
|
||||
<div id="screenshots">
|
||||
${images}
|
||||
</div>`;
|
||||
}
|
||||
}
|
||||
|
||||
customElements.define('rendering-scenario', RenderingScenario);
|
||||
@@ -0,0 +1,74 @@
|
||||
/* @license
|
||||
* Copyright 2019 Google LLC. All Rights Reserved.
|
||||
* Licensed under the Apache License, Version 2.0 (the 'License');
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an 'AS IS' BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
import {Dimensions, ImageComparisonConfig, RendererConfig, ScenarioConfig} from './common.js';
|
||||
|
||||
const defaultScenario = {
|
||||
lighting: '../../../shared-assets/environments/lightroom_14b.hdr',
|
||||
dimensions: {width: 768, height: 768},
|
||||
target: {x: 0, y: 0, z: 0},
|
||||
orbit: {theta: 0, phi: 90, radius: 1},
|
||||
verticalFoV: 45,
|
||||
renderSkybox: false
|
||||
};
|
||||
|
||||
export class ConfigReader {
|
||||
constructor(readonly config: ImageComparisonConfig) {
|
||||
}
|
||||
|
||||
scenarioConfig(name: string): ScenarioConfig|null {
|
||||
const {scenarios} = this.config;
|
||||
|
||||
for (const scenario of scenarios) {
|
||||
if (scenario.name === name) {
|
||||
const output = Object.assign({}, defaultScenario, scenario);
|
||||
|
||||
// This is necessary because Object.assign is not deep
|
||||
output.dimensions =
|
||||
Object.assign({}, defaultScenario.dimensions, scenario.dimensions);
|
||||
output.target =
|
||||
Object.assign({}, defaultScenario.target, scenario.target);
|
||||
output.orbit = Object.assign({}, defaultScenario.orbit, scenario.orbit);
|
||||
return output;
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
rendererConfig(name: string): RendererConfig|null {
|
||||
const {renderers} = this.config;
|
||||
|
||||
for (const renderer of renderers) {
|
||||
if (renderer.name === name) {
|
||||
return renderer;
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
dimensionsForScenario(name: string): Dimensions {
|
||||
const {scenarios} = this.config;
|
||||
|
||||
for (const scenario of scenarios) {
|
||||
if (scenario.name === name) {
|
||||
return scenario.dimensions;
|
||||
}
|
||||
}
|
||||
|
||||
return {width: 0, height: 0};
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,64 @@
|
||||
/* @license
|
||||
* Copyright 2019 Google LLC. All Rights Reserved.
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
|
||||
const canvas: HTMLCanvasElement = document.createElement('canvas');
|
||||
const context: CanvasRenderingContext2D = canvas.getContext('2d')!;
|
||||
|
||||
|
||||
export class ImageAccessor {
|
||||
static fromArrayBuffer(buffer: ArrayBuffer, width: number, height: number):
|
||||
ImageAccessor {
|
||||
return new ImageAccessor(
|
||||
new ImageData(new Uint8ClampedArray(buffer), width, height));
|
||||
}
|
||||
|
||||
static fromImageElement(image: HTMLImageElement): ImageAccessor {
|
||||
canvas.width = image.naturalWidth;
|
||||
canvas.height = image.naturalHeight;
|
||||
context.drawImage(image, 0, 0);
|
||||
return new ImageAccessor(
|
||||
context.getImageData(0, 0, canvas.width, canvas.height));
|
||||
}
|
||||
|
||||
toArrayBuffer(): ArrayBuffer {
|
||||
const {buffer} = this.source.data;
|
||||
return buffer.slice(0, buffer.byteLength) as ArrayBuffer;
|
||||
}
|
||||
|
||||
cssColorAt(x: number, y: number): string {
|
||||
if (x < 0 || y < 0 || x > (this.width - 1) || y > (this.height - 1)) {
|
||||
return 'black';
|
||||
}
|
||||
|
||||
const position = (y * this.width + x) * 4;
|
||||
const array = this.source.data;
|
||||
|
||||
const color = `rgb(${array[position]}, ${array[position + 1]}, ${
|
||||
array[position + 2]})`;
|
||||
return color;
|
||||
}
|
||||
|
||||
constructor(protected source: ImageData) {
|
||||
}
|
||||
|
||||
get width(): number {
|
||||
return this.source.width;
|
||||
}
|
||||
|
||||
get height(): number {
|
||||
return this.source.height;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,131 @@
|
||||
/* @license
|
||||
* Copyright 2019 Google LLC. All Rights Reserved.
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
import {CanvasesReadyMessage, ImageComparator, ImageComparisonMessage, ImagesAssignedMessage, OffscreenCanvas, ThresholdChangedMessage} from './common.js';
|
||||
|
||||
|
||||
class ImageComparisonWorker {
|
||||
protected analyzer: ImageComparator|null = null;
|
||||
protected candidateCanvas: OffscreenCanvas|null = null;
|
||||
protected candidateContext: CanvasRenderingContext2D|null = null;
|
||||
protected goldenCanvas: OffscreenCanvas|null = null;
|
||||
protected goldenContext: CanvasRenderingContext2D|null = null;
|
||||
protected blackWhiteCanvas: OffscreenCanvas|null = null;
|
||||
protected blackWhiteContext: CanvasRenderingContext2D|null = null;
|
||||
protected deltaCanvas: OffscreenCanvas|null = null;
|
||||
protected deltaContext: CanvasRenderingContext2D|null = null;
|
||||
|
||||
constructor() {
|
||||
self.onmessage = (event: MessageEvent) => this.onGlobalMessage(event);
|
||||
}
|
||||
|
||||
onMessage(event: MessageEvent, port: MessagePort): void {
|
||||
const data = event.data as ImageComparisonMessage;
|
||||
|
||||
switch (data.type) {
|
||||
case 'canvases-ready': {
|
||||
const {candidateCanvas, goldenCanvas, blackWhiteCanvas, deltaCanvas} =
|
||||
data as CanvasesReadyMessage;
|
||||
|
||||
this.candidateCanvas = candidateCanvas;
|
||||
this.candidateContext = candidateCanvas.getContext('2d');
|
||||
this.goldenCanvas = goldenCanvas;
|
||||
this.goldenContext = goldenCanvas.getContext('2d');
|
||||
this.blackWhiteCanvas = blackWhiteCanvas;
|
||||
this.blackWhiteContext = blackWhiteCanvas.getContext('2d');
|
||||
this.deltaCanvas = deltaCanvas;
|
||||
this.deltaContext = deltaCanvas.getContext('2d');
|
||||
|
||||
break;
|
||||
}
|
||||
|
||||
case 'images-assigned': {
|
||||
const {candidateImageBuffer, goldenImageBuffer, dimensions} =
|
||||
data as ImagesAssignedMessage;
|
||||
|
||||
if (this.candidateCanvas == null || this.goldenCanvas == null ||
|
||||
this.blackWhiteCanvas == null || this.deltaCanvas == null) {
|
||||
console.warn('Images assigned before canvases are available!');
|
||||
}
|
||||
|
||||
this.candidateCanvas!.width = this.goldenCanvas!.width =
|
||||
this.blackWhiteCanvas!.width = this.deltaCanvas!.width =
|
||||
dimensions.width;
|
||||
this.candidateCanvas!.height = this.goldenCanvas!.height =
|
||||
this.blackWhiteCanvas!.height = this.deltaCanvas!.height =
|
||||
dimensions.height;
|
||||
|
||||
|
||||
const candidateArray = new Uint8ClampedArray(candidateImageBuffer);
|
||||
|
||||
const goldenArray = new Uint8ClampedArray(goldenImageBuffer);
|
||||
const {width, height} = dimensions;
|
||||
|
||||
this.analyzer =
|
||||
new ImageComparator(candidateArray, goldenArray, dimensions);
|
||||
|
||||
this.candidateContext!.putImageData(
|
||||
new ImageData(candidateArray, width, height), 0, 0);
|
||||
|
||||
this.goldenContext!.putImageData(
|
||||
new ImageData(goldenArray, width, height), 0, 0);
|
||||
|
||||
break;
|
||||
}
|
||||
|
||||
case 'threshold-changed': {
|
||||
const {threshold} = data as ThresholdChangedMessage;
|
||||
const {analyzer} = this;
|
||||
|
||||
if (analyzer == null) {
|
||||
console.warn(`Analyzer not created!`);
|
||||
return;
|
||||
}
|
||||
|
||||
const {width, height} = this.analyzer!.dimensions;
|
||||
const result = analyzer.generateVisuals(threshold);
|
||||
|
||||
this.blackWhiteContext!.putImageData(
|
||||
new ImageData(
|
||||
new Uint8ClampedArray(result.imageBuffers.blackWhite!),
|
||||
width,
|
||||
height),
|
||||
0,
|
||||
0);
|
||||
|
||||
this.deltaContext!.putImageData(
|
||||
new ImageData(
|
||||
new Uint8ClampedArray(result.imageBuffers.delta!),
|
||||
width,
|
||||
height),
|
||||
0,
|
||||
0);
|
||||
|
||||
port.postMessage({type: 'analysis-completed', result});
|
||||
break;
|
||||
}
|
||||
|
||||
default:
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
onGlobalMessage(event: MessageEvent): void {
|
||||
event.ports.forEach(
|
||||
port => port.onmessage = (event) => this.onMessage(event, port));
|
||||
}
|
||||
}
|
||||
|
||||
(self as any).imageComparisonWorker = new ImageComparisonWorker();
|
||||
@@ -0,0 +1,15 @@
|
||||
ISC License
|
||||
|
||||
Copyright (c) 2018, Mapbox
|
||||
|
||||
Permission to use, copy, modify, and/or distribute this software for any purpose
|
||||
with or without fee is hereby granted, provided that the above copyright notice
|
||||
and this permission notice appear in all copies.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH
|
||||
REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
|
||||
FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT,
|
||||
INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS
|
||||
OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER
|
||||
TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF
|
||||
THIS SOFTWARE.
|
||||
@@ -0,0 +1,16 @@
|
||||
name: "pixelmatch"
|
||||
description:
|
||||
"JavaScript image comparison library"
|
||||
|
||||
third_party {
|
||||
url {
|
||||
type: HOMEPAGE
|
||||
value: "https://github.com/mapbox/pixelmatch"
|
||||
}
|
||||
url {
|
||||
type: GIT
|
||||
value: "https://github.com/mapbox/pixelmatch.git"
|
||||
}
|
||||
version: "4.0.2"
|
||||
last_upgrade_date { year: 2018 month: 12 day: 18 }
|
||||
}
|
||||
@@ -0,0 +1,58 @@
|
||||
/* @license ISC
|
||||
* @see LICENSE
|
||||
*/
|
||||
// NOTE(cdata): This is an adapted subset of the original pixelmatch library.
|
||||
// The top-level API of the original library has been omitted, as we only make
|
||||
// use of the lower-level features that aren't actually exported by the upstream
|
||||
// module.
|
||||
|
||||
// calculate color difference according to the paper "Measuring perceived color
|
||||
// difference using YIQ NTSC transmission color space in mobile applications" by
|
||||
// Y. Kotsarenko and F. Ramos
|
||||
export function colorDelta(
|
||||
img1: {[index: number]: number},
|
||||
img2: {[index: number]: number},
|
||||
k: number,
|
||||
m: number,
|
||||
yOnly: boolean = false): number {
|
||||
var a1 = img1[k + 3] / 255, a2 = img2[m + 3] / 255,
|
||||
|
||||
r1 = blend(img1[k + 0], a1), g1 = blend(img1[k + 1], a1),
|
||||
b1 = blend(img1[k + 2], a1),
|
||||
|
||||
r2 = blend(img2[m + 0], a2), g2 = blend(img2[m + 1], a2),
|
||||
b2 = blend(img2[m + 2], a2),
|
||||
|
||||
y = rgb2y(r1, g1, b1) - rgb2y(r2, g2, b2);
|
||||
|
||||
if (yOnly)
|
||||
return y; // brightness difference only
|
||||
|
||||
var i = rgb2i(r1, g1, b1) - rgb2i(r2, g2, b2),
|
||||
q = rgb2q(r1, g1, b1) - rgb2q(r2, g2, b2);
|
||||
|
||||
return 0.5053 * y * y + 0.299 * i * i + 0.1957 * q * q;
|
||||
}
|
||||
|
||||
function rgb2y(r: number, g: number, b: number): number {
|
||||
return r * 0.29889531 + g * 0.58662247 + b * 0.11448223;
|
||||
}
|
||||
function rgb2i(r: number, g: number, b: number): number {
|
||||
return r * 0.59597799 - g * 0.27417610 - b * 0.32180189;
|
||||
}
|
||||
function rgb2q(r: number, g: number, b: number): number {
|
||||
return r * 0.21147017 - g * 0.52261711 + b * 0.31114694;
|
||||
}
|
||||
|
||||
// blend semi-transparent color with white
|
||||
function blend(c: number, a: number): number {
|
||||
return 255 + (c - 255) * a;
|
||||
}
|
||||
|
||||
export function drawPixel(
|
||||
output: Uint8Array, pos: number, r: number, g: number, b: number) {
|
||||
output[pos + 0] = r;
|
||||
output[pos + 1] = g;
|
||||
output[pos + 2] = b;
|
||||
output[pos + 3] = 255;
|
||||
}
|
||||
@@ -0,0 +1,141 @@
|
||||
/* @license
|
||||
* Copyright 2019 Google LLC. All Rights Reserved.
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
const ALERT_THRESHOLD = 0.1;
|
||||
|
||||
import {resolve, join} from 'path';
|
||||
const [candidateResultsDirectory, goldenResultsDirectory] =
|
||||
process.argv.slice(2);
|
||||
|
||||
const warn = (message: string) => console.warn(`🚨 ${message}`);
|
||||
const exit = () => {
|
||||
console.log(`📋 Fidelity result comparison concluded`);
|
||||
process.exit(0);
|
||||
};
|
||||
|
||||
console.log(`🛂 Comparing fidelity to previously recorded results at ${
|
||||
goldenResultsDirectory}`);
|
||||
|
||||
const candidateConfig =
|
||||
require(resolve(join(candidateResultsDirectory, 'config.json')));
|
||||
const goldenConfig =
|
||||
require(resolve(join(goldenResultsDirectory, 'config.json')));
|
||||
|
||||
const {scenarios: candidateScenarios, analysisThresholds: candidateThresholds} =
|
||||
candidateConfig;
|
||||
const {scenarios: goldenScenarios, analysisThresholds: goldenThresholds} =
|
||||
goldenConfig;
|
||||
const goldenScenarioMap = new Map();
|
||||
|
||||
const remainingGoldenThresholds = goldenThresholds.slice();
|
||||
|
||||
for (let i = 0; i < candidateThresholds.length; ++i) {
|
||||
const candidateThreshold = candidateThresholds[i];
|
||||
|
||||
if (remainingGoldenThresholds.length === 0) {
|
||||
warn(`${goldenResultsDirectory} tests ${
|
||||
candidateThresholds.length -
|
||||
goldenThresholds.length} fewer thresholds than ${
|
||||
candidateResultsDirectory}`);
|
||||
exit();
|
||||
}
|
||||
|
||||
const goldenThreshold = remainingGoldenThresholds.shift();
|
||||
|
||||
if (candidateThreshold !== goldenThreshold) {
|
||||
warn(`Candidate uses threshold ${
|
||||
candidateThreshold} where golden uses threshold ${goldenThreshold}`);
|
||||
exit();
|
||||
}
|
||||
}
|
||||
|
||||
for (const goldenScenario of goldenScenarios) {
|
||||
goldenScenarioMap.set(goldenScenario.name, goldenScenario);
|
||||
}
|
||||
|
||||
for (const candidateScenario of candidateScenarios) {
|
||||
const {name} = candidateScenario;
|
||||
|
||||
if (!goldenScenarioMap.has(name)) {
|
||||
warn(`${goldenResultsDirectory} does not include scenario "${
|
||||
name}" found in ${candidateResultsDirectory}`);
|
||||
continue;
|
||||
}
|
||||
|
||||
const goldenScenario = goldenScenarioMap.get(name);
|
||||
const candidateAnalysis =
|
||||
require(resolve(join(candidateResultsDirectory, name, 'analysis.json')));
|
||||
const goldenAnalysis =
|
||||
require(resolve(join(goldenResultsDirectory, name, 'analysis.json')));
|
||||
|
||||
const goldenGoldenMap = new Map();
|
||||
|
||||
for (const goldenGolden of goldenScenario.goldens) {
|
||||
goldenGoldenMap.set(goldenGolden.name, goldenGolden);
|
||||
}
|
||||
|
||||
for (let i = 0; i < candidateScenario.goldens.length; ++i) {
|
||||
const candidateGolden = candidateScenario.goldens[i];
|
||||
if (!goldenGoldenMap.has(candidateGolden.name)) {
|
||||
warn(`${goldenResultsDirectory} does not include an analysis of "${
|
||||
candidateGolden.name}" for scenario "${name}" found in ${
|
||||
candidateResultsDirectory}`);
|
||||
continue;
|
||||
}
|
||||
|
||||
const candidateResults = candidateAnalysis.analysisResults[i];
|
||||
const goldenResults = goldenAnalysis.analysisResults[i];
|
||||
|
||||
for (let j = 0; j < candidateThresholds.length; ++j) {
|
||||
const threshold = candidateThresholds[j];
|
||||
|
||||
const candidateThresholdResult = candidateResults[j];
|
||||
const goldenThresholdResult = goldenResults[j];
|
||||
|
||||
for (const key in candidateThresholdResult) {
|
||||
if (!(key in goldenThresholdResult)) {
|
||||
warn(`Golden analysis of "${candidateGolden.name}" in scenario "${
|
||||
name}" is missing metric "${key}"`);
|
||||
continue;
|
||||
}
|
||||
|
||||
const delta =
|
||||
goldenThresholdResult[key] - candidateThresholdResult[key];
|
||||
|
||||
const comparisonDescription =
|
||||
`<model-viewer> <-> ${candidateGolden.name}`;
|
||||
const comparisonConstraints =
|
||||
`"${name}/${key}" @ threshold ${threshold}`;
|
||||
const percentage = `${(delta * 100).toFixed(2)}%`;
|
||||
|
||||
if (delta > ALERT_THRESHOLD) {
|
||||
warn(`${comparisonDescription} ${
|
||||
comparisonConstraints} decreased by ${percentage}!`);
|
||||
} else if (Math.abs(delta) > 0) {
|
||||
const changeDescription = delta > 0 ? 'decreased' : 'increased';
|
||||
|
||||
console.log(`🔍 ${comparisonDescription} ${comparisonConstraints} ${
|
||||
changeDescription} by ${percentage}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
goldenGoldenMap.delete(candidateGolden.name);
|
||||
}
|
||||
|
||||
goldenScenarioMap.delete(name);
|
||||
}
|
||||
|
||||
exit();
|
||||
@@ -0,0 +1,299 @@
|
||||
/* @license
|
||||
* Copyright 2019 Google LLC. All Rights Reserved.
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
import {spawn} from 'child_process';
|
||||
import {promises as fs} from 'fs';
|
||||
import HTTPServer from 'http-server';
|
||||
import module from 'module';
|
||||
import {dirname, join, resolve} from 'path';
|
||||
import yargs from 'yargs';
|
||||
import {hideBin} from 'yargs/helpers';
|
||||
|
||||
import {ArtifactCreator} from '../artifact-creator.js';
|
||||
import {ImageComparisonConfig} from '../common.js';
|
||||
import {ConfigReader} from '../config-reader.js';
|
||||
|
||||
type CommandLineArgs =
|
||||
{
|
||||
config: string,
|
||||
renderer: string[],
|
||||
scenario: string[],
|
||||
port: number,
|
||||
missingOnly: boolean,
|
||||
dryRun: boolean,
|
||||
quiet: boolean
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const argv =
|
||||
await yargs(hideBin(process.argv))
|
||||
.options({
|
||||
'config': {
|
||||
type: 'string',
|
||||
alias: 'c',
|
||||
description: 'Path to configuration json',
|
||||
demandOption:
|
||||
true, // Makes it mandatory. Adjust as per your needs.
|
||||
},
|
||||
'renderer': {
|
||||
type: 'array',
|
||||
alias: 'r',
|
||||
description: 'Limit to specific renderers',
|
||||
demandOption:
|
||||
false, // Makes it mandatory. Adjust as per your needs.
|
||||
},
|
||||
'scenario': {
|
||||
type: 'array',
|
||||
alias: 's',
|
||||
description: 'Limit to specific scenarios',
|
||||
demandOption:
|
||||
false, // Makes it mandatory. Adjust as per your needs.
|
||||
},
|
||||
'port': {
|
||||
type: 'number',
|
||||
alias: 'p',
|
||||
description: 'Port for web server',
|
||||
default: 9040,
|
||||
},
|
||||
'missing-only': {
|
||||
type: 'boolean',
|
||||
alias: 'm',
|
||||
description: 'Only render if an output image is missing',
|
||||
default: false,
|
||||
},
|
||||
'dry-run': {
|
||||
type: 'boolean',
|
||||
alias: 'd',
|
||||
description:
|
||||
'Lists which images would be rendered but doesn\'t render',
|
||||
default: false,
|
||||
},
|
||||
'quiet': {
|
||||
type: 'boolean',
|
||||
alias: 'q',
|
||||
description: 'Hide the puppeteer controlled browser',
|
||||
default: false,
|
||||
},
|
||||
})
|
||||
.help()
|
||||
.alias('help', 'h')
|
||||
.argv;
|
||||
|
||||
const args: CommandLineArgs = {
|
||||
config: argv.config as string,
|
||||
renderer: (argv.renderer || []) as string[],
|
||||
scenario: (argv.scenario || []) as string[],
|
||||
port: argv.port as number,
|
||||
missingOnly: argv['missing-only'],
|
||||
dryRun: argv['dry-run'],
|
||||
quiet: argv.quiet,
|
||||
};
|
||||
|
||||
const require = module.createRequire(import.meta.url);
|
||||
|
||||
const warn = (message: string) => console.warn(`🚨 ${message}`);
|
||||
const exit = (code = 0) => {
|
||||
console.log(`📋 Screenshot updates concluded`);
|
||||
process.exit(code);
|
||||
};
|
||||
|
||||
const configPath = resolve(args.config);
|
||||
const rootDirectory = resolve(dirname(configPath));
|
||||
const config = require(configPath);
|
||||
|
||||
const {renderers} = config;
|
||||
const goldensDirectory = join(rootDirectory, 'goldens');
|
||||
const renderersDirectory = join(rootDirectory, 'renderers');
|
||||
|
||||
let scenarioWhitelist: Set<string>|null = null;
|
||||
let rendererWhitelist: Set<string>|null = null;
|
||||
const rendererList = new Set(config.renderers.map((renderer: any) => {
|
||||
return renderer.name;
|
||||
}));
|
||||
|
||||
// user has specify either scenarios or renderers
|
||||
if (args.renderer.length > 0) {
|
||||
rendererWhitelist = new Set();
|
||||
args.renderer.forEach((rendererName: string) => {
|
||||
if (rendererList.has(rendererName)) {
|
||||
rendererWhitelist!.add(rendererName);
|
||||
} else {
|
||||
warn(`Requested renderer "${rendererName}" not found in config`);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
if (args.scenario.length > 0) {
|
||||
scenarioWhitelist = new Set();
|
||||
args.scenario.forEach((scenarioName: string) => {
|
||||
const scenarioNameLower = scenarioName.toLowerCase();
|
||||
let numMatches = 0;
|
||||
config.scenarios.forEach((scenario: any) => {
|
||||
if (scenario.name.toLowerCase().indexOf(scenarioNameLower) >= 0) {
|
||||
if (!scenarioWhitelist!.has(scenario.name)) {
|
||||
scenarioWhitelist!.add(scenario.name);
|
||||
}
|
||||
numMatches++;
|
||||
}
|
||||
});
|
||||
if (numMatches) {
|
||||
warn(`Requested scenario "${
|
||||
scenarioName}" does not match any names found in config`);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
const run = async (
|
||||
command: string,
|
||||
args: Array<string>,
|
||||
environmentVariables = {},
|
||||
workingDirectory = process.cwd()) =>
|
||||
new Promise<void>((resolve, reject) => {
|
||||
const childProcess = spawn(command, args, {
|
||||
cwd: workingDirectory,
|
||||
env: {...process.env, ...environmentVariables},
|
||||
stdio: ['ignore', 'inherit', 'inherit']
|
||||
});
|
||||
|
||||
childProcess.once('error', (error: any) => {
|
||||
warn(error);
|
||||
});
|
||||
|
||||
childProcess.once('exit', (code) => {
|
||||
if (code === 0) {
|
||||
resolve();
|
||||
} else {
|
||||
reject(new Error('Command failed'));
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
const updateScreenshots = async (config: ImageComparisonConfig) => {
|
||||
const {scenarios} = config;
|
||||
|
||||
console.log(`🆙 Updating screenshots`);
|
||||
|
||||
try {
|
||||
await fs.mkdir(goldensDirectory);
|
||||
} catch (error) {
|
||||
// Ignored...
|
||||
}
|
||||
|
||||
const configReader = new ConfigReader(config);
|
||||
|
||||
const server = HTTPServer.createServer({root: '../../', cache: -1});
|
||||
server.listen(args.port);
|
||||
|
||||
for (const renderer of renderers) {
|
||||
const {
|
||||
name: rendererName,
|
||||
description: rendererDescription,
|
||||
scripts: scripts
|
||||
} = renderer;
|
||||
|
||||
const screenshotCreator = new ArtifactCreator(
|
||||
config,
|
||||
rootDirectory,
|
||||
`http://localhost:${
|
||||
args.port}/packages/render-fidelity-tools/test/renderers/${
|
||||
rendererName}/`);
|
||||
|
||||
for (const scenarioBase of scenarios) {
|
||||
const scenarioName = scenarioBase.name;
|
||||
const scenario = configReader.scenarioConfig(scenarioName);
|
||||
const {model, lighting, dimensions, exclude} = scenario!;
|
||||
const scenarioGoldensDirectory = join(goldensDirectory, scenarioName);
|
||||
const {width, height} = dimensions;
|
||||
|
||||
try {
|
||||
await fs.mkdir(scenarioGoldensDirectory);
|
||||
} catch (error) {
|
||||
// Ignored...
|
||||
}
|
||||
|
||||
if (exclude != null && exclude.includes(rendererName) ||
|
||||
rendererWhitelist != null && !rendererWhitelist.has(rendererName)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
const goldenFilename = `${rendererName}-golden.png`;
|
||||
const rendererDirectory = join(renderersDirectory, rendererName);
|
||||
const goldenPath = join(scenarioGoldensDirectory, goldenFilename);
|
||||
|
||||
if (scenarioWhitelist != null && !scenarioWhitelist.has(scenarioName)) {
|
||||
console.log(`⏭ Skipping ${scenarioName}...`);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (args.missingOnly) {
|
||||
try {
|
||||
await fs.access(goldenPath);
|
||||
console.log(`⏭ Skipping ${scenarioName} as render exists...`);
|
||||
continue;
|
||||
} catch (error) {
|
||||
// ignored
|
||||
}
|
||||
}
|
||||
|
||||
if (scripts != null && scripts.setup != null &&
|
||||
!lighting.includes('spot1Lux')) {
|
||||
const setup = join(rendererDirectory, scripts.setup);
|
||||
console.log(`🚧 Running setup script: ${scripts.setup}`);
|
||||
await run(
|
||||
setup,
|
||||
[],
|
||||
{
|
||||
SCENARIO_NAME: scenarioName,
|
||||
MODEL: resolve(rendererDirectory, model),
|
||||
LIGHTING: resolve(rendererDirectory, lighting),
|
||||
RENDER_WIDTH: width,
|
||||
RENDER_HEIGHT: height
|
||||
},
|
||||
rendererDirectory);
|
||||
}
|
||||
|
||||
if (args.dryRun) {
|
||||
process.stdout.write(
|
||||
rendererName + `: Rendering ` + scenarioName +
|
||||
'... -- skipping, dry-run');
|
||||
continue;
|
||||
}
|
||||
|
||||
try {
|
||||
await screenshotCreator.captureScreenshot(
|
||||
rendererName,
|
||||
scenarioName,
|
||||
dimensions,
|
||||
goldenPath,
|
||||
-1,
|
||||
args.quiet);
|
||||
} catch (error) {
|
||||
throw new Error(`Failed to update ${
|
||||
rendererDescription} screenshot: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
screenshotCreator.close();
|
||||
}
|
||||
server.close();
|
||||
};
|
||||
|
||||
updateScreenshots(config).then(() => exit(0)).catch((error) => {
|
||||
console.error(error);
|
||||
exit(1);
|
||||
});
|
||||
}
|
||||
|
||||
main();
|
||||
@@ -0,0 +1,224 @@
|
||||
/* @license
|
||||
* Copyright 2019 Google LLC. All Rights Reserved.
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
import HTTPServer from 'http-server';
|
||||
import module from 'module';
|
||||
import {dirname, join, resolve} from 'path';
|
||||
import {rimraf} from 'rimraf';
|
||||
import yargs from 'yargs';
|
||||
import {hideBin} from 'yargs/helpers';
|
||||
|
||||
type CommandLineArgs = {
|
||||
config: string; renderer: string; scenario: string[]; port: number;
|
||||
dryRun: boolean;
|
||||
quiet: boolean;
|
||||
}
|
||||
|
||||
|
||||
const require = module.createRequire(import.meta.url);
|
||||
// actions/core can only be imported using commonJS's require here
|
||||
const core = require('@actions/core');
|
||||
|
||||
import {ArtifactCreator} from '../artifact-creator.js';
|
||||
import {FIDELITY_TEST_THRESHOLD, toDecibel} from '../common.js';
|
||||
|
||||
async function main() {
|
||||
const argv =
|
||||
await yargs(hideBin(process.argv))
|
||||
.options({
|
||||
'config': {
|
||||
type: 'string',
|
||||
alias: 'c',
|
||||
description: 'Path to configuration json',
|
||||
demandOption:
|
||||
true, // Makes it mandatory. Adjust as per your needs.
|
||||
},
|
||||
'renderer': {
|
||||
type: 'string',
|
||||
alias: 'r',
|
||||
description: 'Name of web-based renderer to test',
|
||||
default: 'model-viewer',
|
||||
choices: [
|
||||
'filament',
|
||||
'babylon',
|
||||
'gltf-sample-viewer',
|
||||
'three-gpu-renderer',
|
||||
'model-viewer'
|
||||
]
|
||||
},
|
||||
'scenario': {
|
||||
type: 'array',
|
||||
alias: 's',
|
||||
description: 'Limit to specific scenarios',
|
||||
demandOption:
|
||||
false, // Makes it mandatory. Adjust as per your needs.
|
||||
},
|
||||
'port': {
|
||||
type: 'number',
|
||||
alias: 'p',
|
||||
description: 'Port for web server',
|
||||
default: 9030,
|
||||
},
|
||||
'dry-run': {
|
||||
type: 'boolean',
|
||||
alias: 'd',
|
||||
description: 'Checks that all comparison images exist',
|
||||
default: false,
|
||||
},
|
||||
'quiet': {
|
||||
type: 'boolean',
|
||||
alias: 'q',
|
||||
description: 'Hide the puppeteer controlled browser',
|
||||
default: false,
|
||||
},
|
||||
})
|
||||
.help()
|
||||
.alias('help', 'h')
|
||||
.argv;
|
||||
|
||||
const args: CommandLineArgs = {
|
||||
config: argv.config as string,
|
||||
renderer: argv.renderer as string,
|
||||
scenario: (argv.scenario || []) as string[],
|
||||
port: argv.port as number,
|
||||
dryRun: argv['dry-run'],
|
||||
quiet: argv.quiet,
|
||||
};
|
||||
|
||||
const configPath = resolve(args.config);
|
||||
const rootDirectory = resolve(dirname(configPath));
|
||||
const config = require(configPath);
|
||||
|
||||
const outputDirectory = join(rootDirectory, 'results');
|
||||
const screenshotCreator = new ArtifactCreator(
|
||||
config,
|
||||
rootDirectory,
|
||||
`http://localhost:${
|
||||
args.port}/packages/render-fidelity-tools/test/renderers/${
|
||||
args.renderer}/`);
|
||||
const server = HTTPServer.createServer({root: '../../', cache: -1});
|
||||
server.listen(args.port);
|
||||
|
||||
try {
|
||||
rimraf.sync(outputDirectory);
|
||||
} catch (error) {
|
||||
console.warn(error);
|
||||
}
|
||||
|
||||
let scenarioWhitelist: Set<string>|null = null;
|
||||
|
||||
// user has specified scenarios to test
|
||||
if (args.scenario.length > 0) {
|
||||
scenarioWhitelist = new Set();
|
||||
args.scenario.forEach((scenarioName: string) => {
|
||||
const scenarioNameLower = scenarioName.toLowerCase();
|
||||
let numMatches = 0;
|
||||
config.scenarios.forEach((scenario: any) => {
|
||||
if (scenario.name.toLowerCase().indexOf(scenarioNameLower) >= 0) {
|
||||
if (!scenarioWhitelist!.has(scenario.name)) {
|
||||
scenarioWhitelist!.add(scenario.name);
|
||||
}
|
||||
numMatches++;
|
||||
}
|
||||
});
|
||||
if (numMatches) {
|
||||
console.warn(`Requested scenario "${
|
||||
scenarioName}" does not match any names found in config`);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
await screenshotCreator.fidelityTest(
|
||||
scenarioWhitelist, args.renderer, args.dryRun, args.quiet);
|
||||
|
||||
console.log(`✅ Results recorded to ${outputDirectory}`);
|
||||
|
||||
const fidelityRegressionPath =
|
||||
join(outputDirectory, 'fidelityRegressionResults.json');
|
||||
const {
|
||||
results: fidelityRegressionResults,
|
||||
errors: fidelityRegressionErrors,
|
||||
warnings: fidelityRegressionWarnings
|
||||
} = require(fidelityRegressionPath);
|
||||
|
||||
const fidelityRegressionPassedCount = fidelityRegressionResults.length;
|
||||
const fidelityRegressionErrorCount = fidelityRegressionErrors.length;
|
||||
const fidelityRegressionWarningCount = fidelityRegressionWarnings.length;
|
||||
const scenarioCount = fidelityRegressionPassedCount +
|
||||
fidelityRegressionErrorCount + fidelityRegressionWarningCount
|
||||
|
||||
console.log(`\nFidelity regression test on ${
|
||||
scenarioCount} scenarios finished. (Uses ${
|
||||
FIDELITY_TEST_THRESHOLD} dB as threshold)`);
|
||||
console.log(`Passed ${fidelityRegressionPassedCount} scenarios ✅`);
|
||||
if (fidelityRegressionErrorCount > 0) {
|
||||
console.log(`Failed ${fidelityRegressionErrorCount} scenarios ❌`);
|
||||
} else {
|
||||
let maxError = -Infinity;
|
||||
for (const result of fidelityRegressionResults) {
|
||||
maxError = Math.max(maxError, toDecibel(result.rmsDistanceRatio));
|
||||
}
|
||||
console.log('Worst scenario RMS:', maxError, 'dB');
|
||||
}
|
||||
|
||||
if (fidelityRegressionWarningCount > 0) {
|
||||
console.log(`Warnings on ${fidelityRegressionWarningCount} scenarios❗️`);
|
||||
console.log('\n🔍 Logging warning scenarios: ');
|
||||
for (const warning of fidelityRegressionWarnings) {
|
||||
console.log(warning);
|
||||
}
|
||||
|
||||
core.warning(
|
||||
'❗️Fidelity test detected some warnings! Please try to fix them');
|
||||
}
|
||||
|
||||
if (fidelityRegressionErrorCount > 0) {
|
||||
console.log('\n🔍 Logging failed scenarios: ');
|
||||
for (const error of fidelityRegressionErrors) {
|
||||
console.log(error);
|
||||
}
|
||||
}
|
||||
|
||||
const compareRendererResultPath = join(outputDirectory, 'config.json');
|
||||
const {scenarios: comparedRenderResult, errors: compareRendererErrors} =
|
||||
require(compareRendererResultPath);
|
||||
const compareRendererPassCount = comparedRenderResult.length;
|
||||
const compareRendererErrorCount = compareRendererErrors.length;
|
||||
|
||||
if (compareRendererErrorCount > 0) {
|
||||
console.log(
|
||||
`\nCompare Renderers on ${scenarioCount} scenarios finished. ${
|
||||
compareRendererPassCount} scenarios passed ✅, ${
|
||||
compareRendererErrorCount} scenarios failed ❌`);
|
||||
console.log('\n🔍 Logging failed scenarios: ');
|
||||
for (const error of compareRendererErrors) {
|
||||
console.log(error);
|
||||
}
|
||||
}
|
||||
|
||||
if (fidelityRegressionErrorCount > 0 || compareRendererErrorCount > 0) {
|
||||
throw new Error(
|
||||
' ❌ Fidelity test failed! Please fix the errors listed above before merging this pr!');
|
||||
}
|
||||
} catch (error: any) {
|
||||
core.setFailed(error.message);
|
||||
} finally {
|
||||
server.close();
|
||||
screenshotCreator.close();
|
||||
}
|
||||
}
|
||||
|
||||
main();
|
||||
@@ -0,0 +1,62 @@
|
||||
# Fidelity Testing
|
||||
|
||||
## Crafting new test scenarios
|
||||
|
||||
As far as `<model-viewer>` is concerned, a scenario is just an HTML file that
|
||||
incorporates `<model-viewer>` with a given configuration. This HTML file will
|
||||
be loaded by a browser at test time and used to produce a screenshot for
|
||||
comparison to any configured reference renders.
|
||||
|
||||
The steps to produce a new scenario are:
|
||||
|
||||
1. Acquire or produce a glTF that is representative of what the scenario should
|
||||
be testing.
|
||||
- If applicable, also produce a corresponding USDZ of that model
|
||||
2. Pick a descriptive name for the scenario (for example, `alpha-blend-litmus`)
|
||||
3. Create a sub-directory of the [`test/fidelity`](./) folder with the same
|
||||
name as your scenario. This will be the scenario's directory.
|
||||
4. Add any relevant screenshots of reference viewers to the scenario's
|
||||
directory
|
||||
- See above in this document for advice on creating reference viewer
|
||||
screenshots
|
||||
5. Add an `index.html` document containing a configured `<model-viewer>`, and
|
||||
with CSS margins removed from the `<body>`.
|
||||
- Currently, screenshots are taken in an area measured from the top-left of
|
||||
the document, so any margin will most likely lead to undesired results
|
||||
6. Modify the configuration at [`test/fidelity/config.json`](./config.json) to
|
||||
include an item under `scenarios` that corresponds to your newly created
|
||||
scenario
|
||||
- Be sure to specify the dimensions of the test, the set of references
|
||||
(called "goldens") and the thresholds to test at (0, 1 and 10 are
|
||||
recommended).
|
||||
|
||||
|
||||
On execution, a stringyfied scenario configuration will automatically be attached to `args`, the list of arguments to the `executable`. Please note the additional `outputFile` property.
|
||||
|
||||
Example configuration:
|
||||
```json
|
||||
{
|
||||
"scenario": {
|
||||
"lighting": "../../../shared-assets/environments/lightroom_14b.hdr",
|
||||
"dimensions": {
|
||||
"width": 768,
|
||||
"height": 450
|
||||
},
|
||||
"target": {
|
||||
"x": 0,
|
||||
"y": 0.3,
|
||||
"z": 0
|
||||
},
|
||||
"orbit": {
|
||||
"theta": 0,
|
||||
"phi": 90,
|
||||
"radius": 1
|
||||
},
|
||||
"verticalFoV": 45,
|
||||
"renderSkybox": False,
|
||||
"name": "khronos-SheenChair",
|
||||
"model": "../../../shared-assets/models/glTF-Sample-Assets/Models/SheenChair/glTF-Binary/SheenChair.glb"
|
||||
},
|
||||
"outputFile": "./test/goldens/khronos-SheenChair/stellar-golden.png"
|
||||
}
|
||||
```
|
||||
File diff suppressed because it is too large.
Load diff
BIN
Binary file not shown.
|
After Width: | Height: | Size: 227 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 102 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 81 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 168 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 1.6 MiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 1.0 MiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 1.9 MiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 43 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 62 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 15 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 121 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 344 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 412 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 332 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 137 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 1004 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 1.4 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 932 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 493 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 25 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 36 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 25 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 222 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 188 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 252 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 331 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 429 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 300 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 124 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 782 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 513 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 841 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 1.7 MiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 1.6 MiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 1.3 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 287 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 227 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 220 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 909 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 116 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 234 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 354 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 220 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 316 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 36 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 1.2 MiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 737 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 1.6 MiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 579 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 1.1 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 639 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 733 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 771 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 1.5 MiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 1.4 MiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 812 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 843 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 467 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 77 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 1000 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 65 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 199 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 653 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 554 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 162 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 192 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 104 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 44 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 56 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 1.2 MiB |
Loaded 100 of 611 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user