Skip to content

Add JavaScript MIPs for scaling textures larger and smaller #431

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 26, 2019
Merged

Add JavaScript MIPs for scaling textures larger and smaller #431

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions src/SVGMIP.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
const twgl = require('twgl.js');

class SVGMIP {
/**
* Create a new SVG MIP for a given scale.
* @param {RenderWebGL} renderer - The renderer which this MIP's skin uses.
* @param {SvgRenderer} svgRenderer - The svg renderer which this MIP's skin uses.
* @param {number} scale - The relative size of the MIP
* @param {function} callback - A callback that should always fire after draw()
* @constructor
*/
constructor (renderer, svgRenderer, scale, callback) {
this._renderer = renderer;
this._svgRenderer = svgRenderer;
this._scale = scale;
this._texture = null;
this._callback = callback;

this.draw();
Copy link
Contributor

@cwillisf cwillisf Sep 24, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think setting dirty to true by default and skipping this.draw() here might allow this to skip drawing sometimes, or at worst wouldn't hurt.

Copy link
Contributor

@mzgoddard mzgoddard Oct 2, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't have to be for every MIP. But we should at least call draw in setSVG on the 1.0 scale MIP. This "primes" the SVGRenderer by loading an Image instance and drawing it to a canvas. The loading part can need a lot of time, so we should load the svg during the initial project load to avoid potentially large draw times when you switch to a SVG costume that hasn't been drawn before.

}

draw () {
this._svgRenderer._draw(this._scale, () => {
const textureData = this._getTextureData();
const textureOptions = {
auto: false,
wrap: this._renderer.gl.CLAMP_TO_EDGE,
src: textureData
};

this._texture = twgl.createTexture(this._renderer.gl, textureOptions);
this._callback(textureData);
});
}

dispose () {
this._renderer.gl.deleteTexture(this.getTexture());
}

getTexture () {
return this._texture;
}

_getTextureData () {
// Pull out the ImageData from the canvas. ImageData speeds up
// updating Silhouette and is better handled by more browsers in
// regards to memory.
const canvas = this._svgRenderer.canvas;
const context = canvas.getContext('2d');
const textureData = context.getImageData(0, 0, canvas.width, canvas.height);

return textureData;
}
}

module.exports = SVGMIP;
165 changes: 96 additions & 69 deletions src/SVGSkin.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,16 @@
const twgl = require('twgl.js');

const Skin = require('./Skin');
const SVGMIP = require('./SVGMIP');
const SvgRenderer = require('scratch-svg-renderer').SVGRenderer;

const MAX_TEXTURE_DIMENSION = 2048;
const MIN_TEXTURE_SCALE = 1 / 256;
/**
* All scaled renderings of the SVG are stored in an array. The 1.0 scale of
* the SVG is stored at the 8th index. The smallest possible 1 / 256 scale
* rendering is stored at the 0th index.
* @const {number}
*/
const INDEX_OFFSET = 8;

class SVGSkin extends Skin {
/**
Expand All @@ -25,20 +32,28 @@ class SVGSkin extends Skin {
/** @type {WebGLTexture} */
this._texture = null;

/** @type {number} */
this._textureScale = 1;
/** @type {Array.<SVGMIPs>} */
this._scaledMIPs = [];

/** @type {Number} */
this._maxTextureScale = 0;
/**
* Ratio of the size of the SVG and the max size of the WebGL texture
* @type {Number}
*/
this._maxTextureScale = 1;
}

/**
* Dispose of this object. Do not use it after calling this method.
*/
dispose () {
if (this._texture) {
this._renderer.gl.deleteTexture(this._texture);
for (const mip of this._scaledMIPs) {
if (mip) {
mip.dispose();
}
}
this._texture = null;
this._scaledMIPs.length = 0;
}
super.dispose();
}
Expand All @@ -60,11 +75,33 @@ class SVGSkin extends Skin {
super.setRotationCenter(x - viewOffset[0], y - viewOffset[1]);
}

/**
* Create a MIP for a given scale and pass it a callback for updating
* state when switching between scales and MIPs.
* @param {number} scale - The relative size of the MIP
* @param {function} resetCallback - this is a callback for doing a hard reset
* of MIPs and a reset of the rotation center. Only passed in if the MIP scale is 1.
* @return {SVGMIP} An object that handles creating and updating SVG textures.
*/
createMIP (scale, resetCallback) {
const textureCallback = textureData => {
if (resetCallback) resetCallback();
// Check if we have the largest MIP
// eslint-disable-next-line no-use-before-define
if (!this._scaledMIPs.length || this._scaledMIPs[this._scaledMIPs.length - 1]._scale <= scale) {
Copy link
Contributor

@mzgoddard mzgoddard Oct 24, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can make this a little simpler.

One way would be if you pass another argument to createMIP. Pass the scaled index and we can this._scaledMIPs.length - 1 === scaleIndex to test for this.

// Currently silhouette only gets scaled up
this._silhouette.update(textureData);
}
};
const mip = new SVGMIP(this._renderer, this._svgRenderer, scale, textureCallback);

return mip;
}

/**
* @param {Array<number>} scale - The scaling factors to be used, each in the [0,100] range.
* @return {WebGLTexture} The GL texture representation of this skin when drawing at the given scale.
*/
// eslint-disable-next-line no-unused-vars
getTexture (scale) {
if (!this._svgRenderer.canvas.width || !this._svgRenderer.canvas.height) {
return super.getTexture();
Expand All @@ -73,80 +110,70 @@ class SVGSkin extends Skin {
// The texture only ever gets uniform scale. Take the larger of the two axes.
const scaleMax = scale ? Math.max(Math.abs(scale[0]), Math.abs(scale[1])) : 100;
const requestedScale = Math.min(scaleMax / 100, this._maxTextureScale);
let newScale = this._textureScale;
while ((newScale < this._maxTextureScale) && (requestedScale >= 1.5 * newScale)) {
newScale *= 2;
let newScale = 1;
let textureIndex = 0;

if (requestedScale < 1) {
while ((newScale > MIN_TEXTURE_SCALE) && (requestedScale <= newScale * .75)) {
newScale /= 2;
textureIndex -= 1;
}
} else {
while ((newScale < this._maxTextureScale) && (requestedScale >= 1.5 * newScale)) {
newScale *= 2;
textureIndex += 1;
}
}
if (this._textureScale !== newScale) {
this._textureScale = newScale;
this._svgRenderer._draw(this._textureScale, () => {
if (this._textureScale === newScale) {
const canvas = this._svgRenderer.canvas;
const context = canvas.getContext('2d');
const textureData = context.getImageData(0, 0, canvas.width, canvas.height);

const gl = this._renderer.gl;
gl.bindTexture(gl.TEXTURE_2D, this._texture);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, textureData);
this._silhouette.update(textureData);
}
});

if (!this._scaledMIPs[textureIndex + INDEX_OFFSET]) {
this._scaledMIPs[textureIndex + INDEX_OFFSET] = this.createMIP(newScale);
}

return this._texture;
return this._scaledMIPs[textureIndex + INDEX_OFFSET].getTexture();
}

/**
* Set the contents of this skin to a snapshot of the provided SVG data.
* @param {string} svgData - new SVG to use.
* Do a hard reset of the existing MIPs by calling dispose(), setting a new
* scale 1 MIP in this._scaledMIPs, and finally updating the rotationCenter.
* @param {SVGMIPs} mip - An object that handles creating and updating SVG textures.
* @param {Array<number>} [rotationCenter] - Optional rotation center for the SVG. If not supplied, it will be
* calculated from the bounding box
* @fires Skin.event:WasAltered
* @fires Skin.event:WasAltered
*/
setSVG (svgData, rotationCenter) {
this._svgRenderer.fromString(svgData, 1, () => {
const gl = this._renderer.gl;
this._textureScale = this._maxTextureScale = 1;

// Pull out the ImageData from the canvas. ImageData speeds up
// updating Silhouette and is better handled by more browsers in
// regards to memory.
const canvas = this._svgRenderer.canvas;

if (!canvas.width || !canvas.height) {
super.setEmptyImageData();
return;
}
resetMIPs (mip, rotationCenter) {
Copy link
Contributor

@mzgoddard mzgoddard Oct 24, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this method is pretty slick.

this._scaledMIPs.forEach(oldMIP => oldMIP.dispose());
this._scaledMIPs.length = 0;

const context = canvas.getContext('2d');
const textureData = context.getImageData(0, 0, canvas.width, canvas.height);
// Set new scale 1 MIP after outdated MIPs have been disposed
this._texture = this._scaledMIPs[INDEX_OFFSET] = mip;

if (this._texture) {
gl.bindTexture(gl.TEXTURE_2D, this._texture);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, textureData);
this._silhouette.update(textureData);
} else {
// TODO: mipmaps?
const textureOptions = {
auto: true,
wrap: gl.CLAMP_TO_EDGE,
src: textureData
};

this._texture = twgl.createTexture(gl, textureOptions);
this._silhouette.update(textureData);
}
if (typeof rotationCenter === 'undefined') rotationCenter = this.calculateRotationCenter();
Copy link
Contributor

@mzgoddard mzgoddard Oct 2, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In a different change we realized that setting the rotationCenter before the image has drawn will cause a "flash" of the old image at the wrong scale. We need to setRotationCenter after drawing.

One way we might do that is move the rotationCenter set logic in createMIP?

this.setRotationCenter.apply(this, rotationCenter);
this.emit(Skin.Events.WasAltered);
}

const maxDimension = Math.max(this._svgRenderer.canvas.width, this._svgRenderer.canvas.height);
let testScale = 2;
for (testScale; maxDimension * testScale <= MAX_TEXTURE_DIMENSION; testScale *= 2) {
this._maxTextureScale = testScale;
}
/**
* Set the contents of this skin to a snapshot of the provided SVG data.
* @param {string} svgData - new SVG to use.
* @param {Array<number>} [rotationCenter] - Optional rotation center for the SVG.
*/
setSVG (svgData, rotationCenter) {
this._svgRenderer.loadString(svgData);
Copy link
Contributor

@cwillisf cwillisf Sep 24, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of using a dirty flag on the SVGMIP instances, why not just dump all the SVGMIPs here? That might take some extra time (deleting WebGL textures) but unless it's a LOT of time I'd rather have simpler code => less chance of breaking it later :)

Copy link
Contributor

@mzgoddard mzgoddard Oct 2, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could:

  1. Call dispose on each current MIPs.
  2. Set the array length to 0.
  3. Create the 1.0 scale MIP at INDEX_OFFSET.
  4. Force the 1.0 MIP to draw.


if (!this._svgRenderer.canvas.width || !this._svgRenderer.canvas.height) {
super.setEmptyImageData();
return;
}

const maxDimension = Math.ceil(Math.max(this.size[0], this.size[1]));
let testScale = 2;
for (testScale; maxDimension * testScale <= MAX_TEXTURE_DIMENSION; testScale *= 2) {
this._maxTextureScale = testScale;
}

if (typeof rotationCenter === 'undefined') rotationCenter = this.calculateRotationCenter();
this.setRotationCenter.apply(this, rotationCenter);
this.emit(Skin.Events.WasAltered);
});
// Create the 1.0 scale MIP at INDEX_OFFSET.
const textureScale = 1;
const mip = this.createMIP(textureScale, () => this.resetMIPs(mip, rotationCenter));
}

}
Expand Down