From SVG to Canvas and Back

Samuli Kaipiainen

University of Helsinki, Department of Computer Science
samuli.kaipiainen@cs.helsinki.fi

Matti Paksula

Member of W3C SVG Interest Group

University of Helsinki, Department of Computer Science
matti.paksula@cs.helsinki.fi


University of Helsinki
Department of Computer Science

Gustaf Hällströmin katu 2b
00014 University of Helsinki
Helsinki
Finland


Abstract


In this paper we continue our previous work that was published at SVG Open 2009. We identify requirements for using SVG and Canvas effectively by using a trivial drawing application as a starting point. We will show that using browsers native rendering of SVG can be directly used as a bitmap export of SVG. This method has security issues that are briefly explained. Based on the feature requirements and security evaluation, we propose an API method for SVG elements that resembles the toDataURL()-method of Canvas. In addition, we propose a list of enhancements for browser manufacturers that were gathered during this work. Lastly, we list and evaluate SVG compatibility workarounds and combine everything that was presented in this paper to an interactive demonstration page.


Table of Contents

Introduction
Motivation: Complete Drawing Application
Requirements for a Web Based Drawing Application
Our Implementations
Canvas to SVG
SVG to Canvas
Browser Compatibility
Security Issues
Same Origin and Canvas Origin Policy
Experimenting with Origin-Clean and Canvas
Recommendations
SVG.toDataURL() API
For Browser Manufacturers
SVG Compatibility Workarounds
Implementations with Canvas
Implementations with Flash
Problems with Workarounds
Guideline for Drawing Application Technologies
Guideline and the Demonstration Page
Conclusions
Bibliography

Introduction

At SVG Open 2009 we presented a method of using the bitmap content of Canvas element in SVG as an image element. This was done by utilizing canvas.toDataURL()-method that returns a base64-coded pixel representation of the canvas drawing context as a PNG image. This data URI can be set as source for a SVG Image element. In our demonstration we combined the best parts of SVG (vector drawing, native scene graph support) and Canvas (pixel drawing) in one trivial drawing application. SVG to Canvas transfer was done in server-side, by rasterizing the image and then drawing that on a canvas element [Trivial09].

As the method for transferring Canvas to SVG was native in the browser (toDataURL()), the next step is obviously to do the same for SVG to Canvas instead of server-side conversion. It would be tempting to use browsers native rendering, but unfortunately SVG does not support a toDataURL()-method. There has been some discussion about adding same method to SVG, but this has some serious security issues that will be discussed in the "Security Issues" section of this paper.

In this paper we will show an experimental method for using the browsers native rendering support to render SVG on Canvas. We will also present a complete list of the possible methods for using SVG together with Canvas. Evaluation of these methods and application fits is also given. In addition, we present a list of recommendations for browser manufacturers to implement for improved support and adaptation of SVG and Canvas.

Motivation: Complete Drawing Application

Like in the paper we published last year, our motivation is drawing in the web: using the right techniques for the right things. (See "Combining existing components: SVG-Edit and CanvasPaint")

The need for interface between SVG and canvas is real and really required for efficent adaptation of SVG. For example, SVG-edits issue #70, "ability to save canvas as PNG" is currently solved with JavaScript+Canvas rendering workaround although a native implementation would be better [Issue70]. A list of rendering workarounds is given the "SVG Compatibility Workarounds" section of this paper.

Requirements for a Web Based Drawing Application

The following list shows requirements for a completely web based drawing application. This list does not apply only for drawing applications: if every requirement would be supported natively by the browsers, it would make the adaptation of SVG and Canvas more popular.

  1. Pixel drawing
  2. Vector drawing
  3. Interfaces
    1. Bitmap export and import (Canvas to SVG)
    2. Vector rasterization and import (SVG to Canvas)
  4. Exporting
    1. Export vectors (SVG DOM)
    2. Export bitmap (Canvas bitmap)

Previous work shows the best technology fit for the requirements 1, 2 and 3.a, but used server-side rendering for 3.b requiring additional components. In the next section we show and evaluate a method to use browsers native rendering of SVG to rasterize image in order to solve the interface requirement 3.b and exporting of rasterized image requirement 4.b.

Our Implementations

Following two methods allow native, two-way interaction between SVG and Canvas. A proposal for better and standardized implementation for the SVG→Canvas direction is given in the "Recommendations" section of this paper after the security issues associated with it are presented.

Canvas to SVG

This method we presented last year [Trivial09] and is provided here for completeness.

function importCanvas(sourceCanvas, targetSVG) {
    // get base64 encoded png data url from Canvas
    var img_dataurl = sourceCanvas.toDataURL("image/png");

    var svg_img = document.createElementNS(
        "http://www.w3.org/2000/svg", "image");

    svg_img.setAttributeNS(
        "http://www.w3.org/1999/xlink", "xlink:href", img_dataurl);

    targetSVG.appendChild(svg_img);
}

Here we transfer the base64 encoded data URL, given by Canvas.toDataURL, into an SVG Image element. That's all what is needed, and it works in all major browsers, including IE9.

SVG to Canvas

The idea for our SVG→Canvas method is to make use of the web browser rendering of SVG. As the SVG is rasterized in the screen, it should be possible to use those pixels directly in Canvas. However, no direct interface exists (see section "Recommendations"), so we’ll use Canvas for the job. A comment on Ajaxian motivated us in this work.

function importSVG(sourceSVG, targetCanvas) {
    // https://developer.mozilla.org/en/XMLSerializer
    svg_xml = (new XMLSerializer()).serializeToString(sourceSVG);
    var ctx = targetCanvas.getContext('2d');

    // this is just a JavaScript (HTML) image
    var img = new Image();
    // http://en.wikipedia.org/wiki/SVG#Native_support
    // https://developer.mozilla.org/en/DOM/window.btoa
    img.src = "data:image/svg+xml;base64," + btoa(svg_xml);

    img.onload = function() {
        // after this, Canvas’ origin-clean is DIRTY
        ctx.drawImage(img, 0, 0);
    }
}

Web browsers only expose the SVG data in DOM, which can also be serialized to XML. We used this in your last years implementation where we sent the serialized XML to server-side rasterizer, which returned a PNG image (with transparency).

Canvas allows to draw arbitrary images on it via the drawImage-function. The Canvas specification for drawImage() says: "The image argument is an instance of either HTMLImageElement, HTMLCanvasElement, or HTMLVideoElement." So any image that the browser supports, can be drawn on Canvas [Canvas10].

All browsers support images with data URI [RFC2397]. Currently, some browsers (Webkit) support SVG images as a source for img elements [SVGimg10]. This feature is also in the test suite for Internet Explorer 9 [MSDNTest]. By combining these, we can transfer the serialized XML SVG data into a HTML Image element. This HTML Image element can be used as an argument for the drawImage()-method of Canvas.

Currently (2010-08) this only works in Safari and Chrome. Also, the SVG→Canvas transfer is write-only, as it triggers the origin-clean flag of Canvas as dirty, which causes that Canvas to be unreadable (until reloaded and cleared). This security issue will be discussed in the "Security Issues" section.

Further, there is no guarantee that this method renders the same image, as currently visible on SVG element. If there is SMIL animation on SVG, the animation might reset to first frame on the transferred image.

Browser Compatibility

Techniques used in the Canvas→SVG transfer are better and more consistently supported than for the SVG→Canvas transfer, as can be seen from the quirk table below.

Table 1: Browser compatibility for Canvas-SVG interaction
Quirk / Browser Safari 4.1 (Webkit) Chrome 5.0 (Webkit) Firefox 3.6 Opera 10.60 IE 9 Preview
XML-serialize dynamic SVG data ok ok ok ok fail
Non-standard btoa() base64 function ok ok ok ok fail
Data URL in HTML img element ok ok ok ok ok
SVG in HTML img element ok ok fail 1 ok ok
SVG img for Canvas drawImage() 2 ok ok fail fail fail
...above should set origin-clean dirty ok ok
Canvas.toDataURL() ok ok ok ok ok
Right-click on Canvas allows "Save as (png)…" not not ok not not
Right-click on SVG allows "Save as (svg)…" not not not not ok

Security Issues

Transferring data from SVG to Canvas has security issues, which cause Canvas to become write-only. We argue that these issues could be avoided with our SVG.toDataURL() proposal (section "Recommendations").

Same Origin and Canvas Origin Policy

Web pages are composed of different elements coming from different origins. Elements coming from the same origin are considered to be safe [Origin10].

Canvas has powerful image reading and writing capabilities. It would be trivial to use canvas as middleman for transfering a local image to a third-party just by loading image into Canvas element from file:// -URL and then sending the pixel data from the Canvas element to any host with JavaScript.

To prevent information leakage with Canvas, browsers are carefully protecting the usage of Canvas when the source for image data is not safe. All Canvas elements are created as their origin-clean attribute set to true. When any of the actions that may potentially be used for using Canvas element to transfer content that violates the same origin policy, the origin-clean property is permanently set to false.

If methods that return the pixel data stored in canvas, such as toDataURL() or getImageData(), are called on the Canvas element whose origin-clean is false, then a DOMException 18 SECURITY_ERR is raised [Canvas10].

Experimenting with Origin-Clean and Canvas

To get better understanding of origin-clean behaviour, we evaluated following cases.

PNG and drawImage()

We test following two trivial cases first:

Case 1a: load a PNG image to the DOM, draw that on Canvas element with drawImage()-method.

Case 1b: load a PNG image to the DOM, create an instance of JavaScript Image-object and draw that on Canvas element with drawImage()-method

In 1a and 1b, the Canvas element stays origin-clean as expected. JavaScript Image-object, as long as the MIME type is image/png works also. It seems like all MIME types other than image/svg+xml works this way.

Case 1c: Load an image from local file://-URL and display it on HTML document loaded from file://-URL. Then draw that image on Canvas element with drawImage()-method.

In 1c the Canvas element is not origin-clean. No data can be exported from Canvas, any attempts will cause a SECURITY_ERR exception. This the standard behavior that was changed lately in all browsers, like in Mozilla Gecko [MozillaSameOrigin].

PNG as Data URI

When a PNG is represented as data URI so that img-elements source attribute is set to data URI: img.src = "data:image/png;base64,iVBORw0..." and this img-element is rendered on Canvas with drawImage()-method, the canvas stays origin-clean.

SVG to Canvas

If the source attribute of HTML img-element is set to "shapes.svg", and this image element is rendered on Canvas element, the origin-clean is set to false even if same origin policy should be satisfied.

SVG as Data URI

If the SVG is represented as a data URI: "data:image/svg+xml;base64,dVAx..." and this string is set as source attribute of a HTML image, which is then rendered on Canvas element, the origin-clean sets to false.

SVG with Safe Elements

This testing becomes more interesting if we do the same tests with a SVG that uses only safe elements, such as <rect> and <circle>. Result will be that the Canvas origin-clean property sets permanently to false, even if all the elements can be considered as "safe" and same origin.

Conclusions on SVG Security Issues

So, why SVG is dangerous? SVG can potentially contain content from multiple origins and browsers tend to blacklist any content with the MIME type image/svg+xml as multi-origin content without even testing the actual content. A good summary of this is given in the comments of Webkit bug #29305.

SVG that contains only safe and same-origin elements should however be rendered to allow efficent use of SVG with Canvas. In the next section we propose a recommendation for a SVG.toDataURL()-method that allows bitmap export of the vector elements in SVG.

Recommendations

SVG.toDataURL() API

Based on the use cases stated as our motivation (section Requirements for a Web Based Drawing Application), and the security discussion in previous section, we propose a following API to allow exporting the rendered bitmap content from SVG.

SVG.toDataURL( [type], [keepNonSafe=false], [keepOutsideViewport=false] )
type
MIME type of the exported data.
Default: image/svg+xml.
Must support: image/png.
keepNonSafe
Export non-safe (image and foreignObject) elements. This will set the Canvas origin-clean property to false, if the data is transferred to Canvas.
Default: false (to keep origin-clean true).
keepOutsideViewport
Export all drawn content, even if not visible.
Default: false, export only visible viewport, similar to Canvas toDataURL().

Implementation with JavaScript

We implemented this in our reference implementation, SVG.toDataURL.js. The library adds toDataURL()-method in the prototype of SVGElement. For better compatibility our library provides XML serialization and base64-encoding for Internet Explorer, as XMLSerializer and btoa are not provided by the browser itself.

We're waiting for a better implementation by browser manufacturers.

For Browser Manufacturers

In addition to the SVG.toDataURL() method presented above, we would like to see following SVG-Canvas-cooperation improvements in browsers.

Save SVG as image

Right-clicking on SVG element should allow saving as image (SVG or PNG). Some browsers already support this on Canvas, but no-one on SVG.

Safari, Chrome (Webkit) and Opera

Right-clicking Canvas should prompt for "save as" (PNG). This is the easiest way to export anything that has been drawn on Canvas.

Firefox

Support for SVG files as source for HTML img elements (Bug 276431).

Internet Explorer

An XML serializer (XMLSerializer) is needed for the browser to allow easy exporting of dynamic SVG data.

SVG Compatibility Workarounds

SVG 1.1 recommendation is seven years old, but still not very well supported [SvgSupport]. Good statistics on SVG support can be seen at http://www.codedread.com/svg-support.php. To address the lacking support for SVG and the security issue presented above, a great number of projects has spawned trying to implement SVG rendering with other technologies.

Alternative technologies to render SVG are Canvas, Flash and VML. In the following list we have tried to gather all compatibility workarounds and other related projects of this field. At the end of this section some notes on problems associated with the workarounds are given.

CanVG

http://code.google.com/p/canvg/

CanVG is the most feature complete SVG compatibility JavaScript library for browsers that support Canvas. The projects end goal is to support everything in the SVG specification. And currently CanVG is selected for the SVG-Edit to solve SVG rasterization. We will evaluate the usage of CanVG later in this paper.

SVG Web

http://code.google.com/p/svgweb/

SVG Web renders SVG files with Adobe Flash. At the time of writing it supports ~50% of the SVG 1.1 specification [SvgSupport].

CakeJS

http://code.google.com/p/cakejs/wiki/Documentation#SVGParser

The Canvas Animation JavaScript library CakeJS has implemented SVG parser that converts SVG DOM to its own set of drawing commands that can be used to rasterize and animate SVG elements on Canvas.

Raphaël, Dojox.gfx and SVGKit SVGCanvas

http://raphaeljs.com/
http://docs.dojocampus.org/dojox/gfx
http://svgkit.sourceforge.net/SVGCanvas.html

Raphaël and dojox.gfx do not implement SVG compatibility, but rather provide own API that can be used for drawing vector graphics. Raphaël is a JavaScript library that provides Canvas like API for drawing graphics. It uses SVG for drawing graphics and falls back to VML if the browser does not support SVG.

Dojox.gfx is an extension to the Dojo toolkit that similary to Raphaël uses SVG and VML, but also Silverlight and Canvas for drawing vector graphics.

SVGKits SVGCanvas is also mentioned, because it provides Canvas API implementation that is rendered with SVG.

Flanvas

http://flanvas.com/

"Flanvas provides a way of interactive development similar to flex. The hope is that actionscript developers will find this to be an easy solution to use when flash is either too much, or not available."

Flanvas parses SVG files and creates own representation of them that can be animated. Support for SVG is still limited.

Abandoned CanvaSVG

http://web.archive.org/web/20070928172356/http%3a//fuchsia-design.com/CanvaSVG/

In order this listing to be more or less complete, it’s important to include already abandoned hack project to stress the fact that Canvas is very lucrative platform for developers to start implementing SVG support.

PNG Rasterization

Static SVGs can also be rasterized as PNG images that are displayed instead of the SVG files. This allows modifications still to be done on SVGs while providing full browser support. Wikipedia uses this technique to display their great amount of SVG images: http://upload.wikimedia.org/wikipedia/commons/thumb/f/f2/Butterfly.svg/398px-Butterfly.svg.png.

Problems with Workarounds

There should not be any need to re-implement, but while native rendering support is missing, the following aspects should be considered.

Scene Graph is Missing with Canvas

In order to have interactivity with "vectorized" elements on Canvas, the implementation needs to keep track of the elements, their boundaries and ordering. This brings some overhead as discussed in our previous paper [Trivial09].

Accessibility

More important than scene graph is the accessibility of the web. Especially with workaround solutions where the source is not available as SVG, but only as a non-accessible bitmap the accessibility is very limited.

Speed and Complexity

Not only the full support of SVG is quite complex task to complete with JavaScript and Canvas or Flash, but it is also more likely to be slower than a native implementation.

Guideline for Drawing Application Technologies

Based on the requirements stated above, here are our implementation recommendations for the current state of technologies. This is due to change in future, when better solutions (SVG.toDataURL) become available.

Guideline and the Demonstration Page

See our demonstration page here: http://db.cs.helsinki.fi/~paksula/svg_open_2010/demo.xhtml

Our demonstration page shows a list of feature requirements for a complete drawing application using SVG and Canvas. Each of the requirements are implemented with most suitable methods. As the method used might make Canvas write-only, the page warns you if this would happen (orange background) and if it has happened (gray background). Refreshing the page resets the status.

  1. Draw pixel graphics with Canvas
  2. Draw vector graphics with SVG
  3. Interfaces between Canvas and SVG:
    1. One-way only transfer from Canvas: canvas → svg, canvas → svg, ...
      1. Canvas.toDataURL(). Can be used multiple times when no SVG data is written onto Canvas.
    2. One-way only transfer from SVG: svg → canvas, svg → canvas, ...
      1. Our client-side implementation. Canvas becomes write-only.
    3. Two-way transfer: svg → canvas → svg → canvas → svg ...
      1. Canvas.toDataURL(). (as in 3.a.i)
      2. SVG → Canvas complete (non-safe) transfer, including possibly non-secure elements:
        1. SVG rastrerization server and Ajax (with Imagemagick or similar).
        2. Our proposed SVG.toDataURL() recommendation. See our JavaScript library.
      3. SVG → Canvas security-safe (non-image and non-foreign) elements:
        1. Our proposed SVG.toDataURL() recommendation. See our JavaScript library.
        2. Canvg (or other SVG rasterization library). Will make Canvas write-only only if the imported SVG has images from different origin, or embedded svg images. Canvg paints transparent areas from SVG as white, overwriting previous drawings on Canvas.
  4. Exporting as file
    1. SVG → PNG
      1. Our proposed SVG.toDataURL() recommendation. See our JavaScript library.
      2. Serverside (safe and non-safe elements)
      3. Canvg (or other SVG rasterization library), safe elements
    2. Canvas → PNG
      1. Right-click → Save as.. (Supported only by Firefox)
      2. Canvas.toDataURL (img element / new window)
    3. SVG → SVG file
      1. Right-click → Save as.. (supported only by IE9)
      2. Serialize SVG as data URL, see SVG.toDataURL() (img element / new window)

Conclusions

The future web, based on open HTML5 standards, is not complete without fully supported vector graphics. Since the last SVG Open 2009, the popularity of SVG has not raised, but the support has become better, especially with the latest Internet Explorer 9 preview releases.

The list of compatibility libraries in this paper show that a) SVG rendering API support has to get better, and b) the interaction with different HTML5 technologies, like Canvas, need to improve as well. Supporting better interaction with SVG, such as our proposed toDataURL()-method, would make SVG less isolated. As SVG has powerful features, the security issues that come with SVG need to be understood. The current security model is too tight for effective interaction with SVG.

Bibliography

[Trivial09] Samuli Kaipiainen and Matti Paksula: SVG vs Canvas on Trivial Drawing Application.
Proceedings of the 7th International Conference on Scalable Vector Graphics
http://svgopen.org/2009/papers/54-SVG_vs_Canvas_on_Trivial_Drawing_Application/

[Issue70] SVG-Edit Issue #70: Ability to save canvas as PNG.
SVG-Edit Project
http://code.google.com/p/svg-edit/issues/detail?id=70

[Canvas10] HTML5 Draft Standard 20: The canvas element, August 2010.
Web Hypertext Application Technology Working Group
http://www.whatwg.org/specs/web-apps/current-work/multipage/the-canvas-element.html

[RFC2397] RFC2397 The "data" URL scheme, August 1998.
The Internet Engineering Task Force (IETF)
http://www6.ietf.org/rfc/rfc2397.txt

[SVGimg10] Wikipedia SVG: Native Support.
http://en.wikipedia.org/wiki/SVG#Native_support

[Origin10] HTML5 Draft Standard 20: Origin, August 2010.
Web Hypertext Application Technology Working Group
http://www.whatwg.org/specs/web-apps/current-work/multipage/origin-0.html#same-origin

[MozillaSameOrigin] Same-origin policy for file: URIs
Mozilla Developer Center
https://developer.mozilla.org/En/Same-origin_policy_for_file:_URIs

[SvgSupport] SVG Support tables
Codedread
http://www.codedread.com/svg-support.php

[MSDNTest] IE9 Testing Center, Canvas drawImage with SVG
Microsoft
http://samples.msdn.microsoft.com/ietestcenter/html5/canvas_harness.htm?url=canvas-images-drawImage-001

[MotvationComment] Comment by bytestrom.eu in Ajaxian
bytestrom.eu in Ajaxian
http://ajaxian.com/archives/todataurl-canvas-and-svg#comment-275765