[IMP] website: add possibility to self host google font

Before this commit, the only possibility when adding a google font was to use google servers to serve the font. This was not ideal as some people really want to serve it themselves without the need of their visitors to reach google servers. That's especially true since recently where it seems like German clients are receiving letters about that to tell them it's illegal and this should be changed, as it wouldn't respect the GDPR. Somehow, it seems related to the fact that google knows you visited a website by just downloading the font, because they very well know with just your IP who you are exactly. It's yet unsure if that issue is well-founded or not, but since German courts seem to be sanctioning people about this, there is no reason to not at least provide a workaround. What is sure is that it makes a lot of noise and more and more people seem to be impacted by this as many opw are getting opened, as well as github messages. Whether it is well-founded or not is thus not really our problem anymore, we should just provide a way for our users to protect themselves against this "German law problem" (or at least think they are protecting, if Odoo thinks that's a non issue or the German court is wrong or ambiguous). Note that a cookies banner to inform users would not be enough for that "problem", as the user would already have accessed your website and thus the related problematic fonts. Another solution which is not something we want (at all) would be to serve local system fonts while the user did not consent about google fonts, or having a blocking screen page telling people visiting the website will fetch google fonts. Obviously those 2 possibilities are a no go as it leads to terrible UX. Finally, note that: - in Odoo 16, the default fonts will be the system fonts, meaning there won't be any call to google by default, regardless of this pr - there is a work in progress to improve the current cookies bar to differentiate essential and non essential cookies and to allow user to accept only one or both (task-2800976). Useful links: - https://github.com/odoo/odoo/issues/83638#issuecomment-1054470699 ODO detailed point of view about this - https://rewis.io/urteile/urteil/lhm-20-01-2022-3-o-1749320/ The German law about this Closes #83638 task-2756486 opw-2970167 opw-2960466 opw-2960555 opw-2952427 opw-2800976 opw-2748647 (possibly many more) Courtesy of @bso-odoo for the regex part which was inspired by another of his google font fix attempt closes odoo/odoo#101826 X-original-commit: b06ce21eba6388ce34bbffffadcb489f0e8557dd Signed-off-by: Quentin Smetz (qsm) <qsm@odoo.com> Signed-off-by: Romain Derie (rde) <rde@odoo.com>
2022-10-02 16:26:40 +02:00
parent b6a0e89371
commit 340d4a09c3
8 changed files with 151 additions and 7 deletions
@@ -1,7 +1,11 @@
 # -*- coding: utf-8 -*-
 # Part of Odoo. See LICENSE file for full copyright and licensing details.

+import base64
 import re
+import requests
+
+from werkzeug.urls import url_parse

 from odoo import api, models

@@ -26,6 +30,7 @@ class Assets(models.AbstractModel):
                word hook). If a key is already in the file's map, its value is
                overridden.
        """
+        IrAttachment = self.env['ir.attachment']
        if 'color-palettes-name' in values:
            self.reset_asset('/website/static/src/scss/options/colors/user_color_palette.scss', 'web.assets_frontend')
            self.reset_asset('/website/static/src/scss/options/colors/user_gray_color_palette.scss', 'web.assets_frontend')
@@ -44,6 +49,76 @@ class Assets(models.AbstractModel):
                'copyright-gradient': 'null',
            })

+        delete_attachment_id = values.pop('delete-font-attachment-id', None)
+        if delete_attachment_id:
+            delete_attachment_id = int(delete_attachment_id)
+            IrAttachment.search([
+                '|', ('id', '=', delete_attachment_id),
+                ('original_id', '=', delete_attachment_id),
+                ('name', 'like', '%google-font%')
+            ]).unlink()
+
+        google_local_fonts = values.get('google-local-fonts')
+        if google_local_fonts and google_local_fonts != 'null':
+            # "('font_x': 45, 'font_y': '')" -> {'font_x': '45', 'font_y': ''}
+            google_local_fonts = dict(re.findall(r"'([^']+)': '?(\d*)", google_local_fonts))
+            # Google is serving different font format (woff, woff2, ttf, eot..)
+            # based on the user agent. We need to get the woff2 as this is
+            # supported by all the browers we support.
+            headers_woff2 = {
+                'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.41 Safari/537.36',
+            }
+            for font_name in google_local_fonts:
+                if google_local_fonts[font_name]:
+                    google_local_fonts[font_name] = int(google_local_fonts[font_name])
+                else:
+                    font_family_attachments = IrAttachment
+                    font_content = requests.get(
+                        f'https://fonts.googleapis.com/css?family={font_name}&display=swap',
+                        timeout=5, headers=headers_woff2,
+                    ).content.decode()
+
+                    def fetch_google_font(src):
+                        statement = src.group()
+                        url, font_format = re.match(r'src: url\(([^\)]+)\) (.+)', statement).groups()
+                        req = requests.get(url, timeout=5, headers=headers_woff2)
+                        # https://fonts.gstatic.com/s/modak/v18/EJRYQgs1XtIEskMB-hRp7w.woff2
+                        # -> s-modak-v18-EJRYQgs1XtIEskMB-hRp7w.woff2
+                        name = url_parse(url).path.lstrip('/').replace('/', '-')
+                        attachment = IrAttachment.create({
+                            'name': f'google-font-{name}',
+                            'type': 'binary',
+                            'datas': base64.b64encode(req.content),
+                            'public': True,
+                        })
+                        nonlocal font_family_attachments
+                        font_family_attachments += attachment
+                        return 'src: url(/web/content/%s/%s) %s' % (
+                            attachment.id,
+                            name,
+                            font_format,
+                        )
+
+                    font_content = re.sub(r'src: url\(.+\)', fetch_google_font, font_content)
+
+                    attach_font = IrAttachment.create({
+                        'name': f'{font_name} (google-font)',
+                        'type': 'binary',
+                        'datas': base64.encodebytes(font_content.encode()),
+                        'mimetype': 'text/css',
+                        'public': True,
+                    })
+                    google_local_fonts[font_name] = attach_font.id
+                    # That field is meant to keep track of the original
+                    # image attachment when an image is being modified (by the
+                    # website builder for instance). It makes sense to use it
+                    # here to link font family attachment to the main font
+                    # attachment. It will ease the unlink later.
+                    font_family_attachments.original_id = attach_font.id
+
+            # {'font_x': 45, 'font_y': 55} -> "('font_x': 45, 'font_y': 55)"
+            values['google-local-fonts'] = str(google_local_fonts).replace('{', '(').replace('}', ')')
+
        custom_url = self._make_custom_asset_url(url, 'web.assets_frontend')
        updatedFileContent = self._get_content_from_url(custom_url) or self._get_content_from_url(url)
        updatedFileContent = updatedFileContent.decode('utf-8')
@@ -116,6 +116,9 @@ const FontFamilyPickerUserValueWidget = SelectUserValueWidget.extend({
        const googleFontsProperty = weUtils.getCSSVariableValue('google-fonts', style);
        this.googleFonts = googleFontsProperty ? googleFontsProperty.split(/\s*,\s*/g) : [];
        this.googleFonts = this.googleFonts.map(font => font.substring(1, font.length - 1)); // Unquote
+        const googleLocalFontsProperty = weUtils.getCSSVariableValue('google-local-fonts', style);
+        this.googleLocalFonts = googleLocalFontsProperty ?
+            googleLocalFontsProperty.slice(1, -1).split(/\s*,\s*/g) : [];

        await this._super(...arguments);

@@ -136,14 +139,25 @@ const FontFamilyPickerUserValueWidget = SelectUserValueWidget.extend({
            this.menuEl.appendChild(fontEl);
        });

+        if (this.googleLocalFonts.length) {
+            const googleLocalFontsEls = fontEls.splice(-this.googleLocalFonts.length);
+            googleLocalFontsEls.forEach((el, index) => {
+                $(el).append(core.qweb.render('website.delete_google_font_btn', {
+                    index: index,
+                    local: true,
+                }));
+            });
+        }
+
        if (this.googleFonts.length) {
-            const googleFontsEls = fontEls.slice(-this.googleFonts.length);
+            const googleFontsEls = fontEls.splice(-this.googleFonts.length);
            googleFontsEls.forEach((el, index) => {
                $(el).append(core.qweb.render('website.delete_google_font_btn', {
                    index: index,
                }));
            });
        }
+
        $(this.menuEl).append($(core.qweb.render('website.add_google_font_btn', {
            variable: variable,
        })));
@@ -217,10 +231,16 @@ const FontFamilyPickerUserValueWidget = SelectUserValueWidget.extend({
                        }

                        const font = m[1].replace(/\+/g, ' ');
-                        this.googleFonts.push(font);
+                        const googleFontServe = dialog.el.querySelector('#google_font_serve').checked;
+                        if (googleFontServe) {
+                            this.googleFonts.push(font);
+                        } else {
+                            this.googleLocalFonts.push(`'${font}': ''`);
+                        }
                        this.trigger_up('google_fonts_custo_request', {
                            values: {[variable]: `'${font}'`},
                            googleFonts: this.googleFonts,
+                            googleLocalFonts: this.googleLocalFonts,
                        });
                    },
                },
@@ -238,6 +258,7 @@ const FontFamilyPickerUserValueWidget = SelectUserValueWidget.extend({
     */
    _onDeleteGoogleFontClick: async function (ev) {
        ev.preventDefault();
+        const values = {};

        const save = await new Promise(resolve => {
            Dialog.confirm(this, _t("Deleting a font requires a reload of the page. This will save all your changes and reload the page, are you sure you want to proceed?"), {
@@ -251,15 +272,23 @@ const FontFamilyPickerUserValueWidget = SelectUserValueWidget.extend({

        // Remove Google font
        const googleFontIndex = parseInt(ev.target.dataset.fontIndex);
-        const googleFont = this.googleFonts[googleFontIndex];
-        this.googleFonts.splice(googleFontIndex, 1);
+        const isLocalFont = ev.target.dataset.localFont;
+        let googleFontName;
+        if (isLocalFont) {
+            const googleFont = this.googleLocalFonts[googleFontIndex].split(':');
+            googleFontName = googleFont[0];
+            values['delete-font-attachment-id'] = googleFont[1];
+            this.googleLocalFonts.splice(googleFontIndex, 1);
+        } else {
+            googleFontName = this.googleFonts[googleFontIndex];
+            this.googleFonts.splice(googleFontIndex, 1);
+        }

        // Adapt font variable indexes to the removal
-        const values = {};
        const style = window.getComputedStyle(document.documentElement);
        _.each(FontFamilyPickerUserValueWidget.prototype.fontVariables, variable => {
            const value = weUtils.getCSSVariableValue(variable, style);
-            if (value.substring(1, value.length - 1) === googleFont) {
+            if (value.substring(1, value.length - 1) === googleFontName) {
                // If an element is using the google font being removed, reset
                // it to the theme default.
                values[variable] = 'null';
@@ -269,6 +298,7 @@ const FontFamilyPickerUserValueWidget = SelectUserValueWidget.extend({
        this.trigger_up('google_fonts_custo_request', {
            values: values,
            googleFonts: this.googleFonts,
+            googleLocalFonts: this.googleLocalFonts,
        });
    },
 });
@@ -722,11 +752,17 @@ options.Class.include({
    _onGoogleFontsCustoRequest: function (ev) {
        const values = ev.data.values ? _.clone(ev.data.values) : {};
        const googleFonts = ev.data.googleFonts;
+        const googleLocalFonts = ev.data.googleLocalFonts;
        if (googleFonts.length) {
            values['google-fonts'] = "('" + googleFonts.join("', '") + "')";
        } else {
            values['google-fonts'] = 'null';
        }
+        if (googleLocalFonts.length) {
+            values['google-local-fonts'] = "(" + googleLocalFonts.join(", ") + ")";
+        } else {
+            values['google-local-fonts'] = 'null';
+        }
        this.trigger_up('snippet_edition_request', {exec: async () => {
            return this._makeSCSSCusto('/website/static/src/scss/options/user_values.scss', values);
        }});
@@ -1954,6 +1954,7 @@ $o-selected-color-palettes-names: (
 $o-base-website-values-palette: (
    'font-size-base': 1rem, // Need a set value as the value is used in bootstrap_overridden files
    'google-fonts': null,
+    'google-local-fonts': null,

    'body-image': null,
    'body-image-type': 'image', // 'image' or 'pattern'
@@ -230,6 +230,17 @@ $o-theme-font-configs: map-merge($o-base-theme-font-configs, $o-theme-font-confi
    ));
 }

+// Add locally hosted google fonts
+@each $font-name, $font-attach-id in (o-website-value('google-local-fonts') or ()) {
+    $o-theme-font-configs: map-merge($o-theme-font-configs, (
+        $font-name: (
+            'family': (quote($font-name), sans-serif),
+            'attachment': $font-attach-id,
+            'name': quote($font-name),
+        ),
+    ));
+}
+
 // Add odoo unicode support for all fonts
@each $font-name, $font-config in $o-theme-font-configs {
    $o-theme-font-configs: map-merge($o-theme-font-configs, (
@@ -8,6 +8,12 @@ $-seen-urls: ();
    @if $-url and index($-seen-urls, $-url) == null {
        $-seen-urls: append($-seen-urls, $-url);
        @import url("https://fonts.googleapis.com/css?family=#{unquote($-url)}&display=swap");
+    } @else {
+        $-name: o-get-font-info($alias, 'name');
+        $-attachment: o-get-font-info($alias, 'attachment');
+        @if $-attachment {
+            @import url("/web/content/#{$-attachment}/google-font-#{unquote($-name)}");
+        }
    }
 }

@@ -3,6 +3,12 @@
    $url: map-get($font-config, 'url');
    @if $url {
        @import url("https://fonts.googleapis.com/css?family=#{unquote($url)}&display=swap");
+    } @else {
+        $name: map-get($font-config, 'name');
+        $attachment: map-get($font-config, 'attachment');
+        @if $attachment {
+            @import url("/web/content/#{$attachment}/google-font-#{unquote($name)}");
+        }
    }
 }

@@ -61,15 +61,21 @@
                    Select one font on <a target="_blank" href="https://fonts.google.com">fonts.google.com</a> and copy paste the address of the font page here.
                </span>
            </div>
+            <label class="col-form-label col-md-3" for="google_font_serve">Serve font from Google servers</label>
+            <label class="o_switch col-form-label col-md-9" for="google_font_serve">
+                <input type="checkbox" checked="checked" id="google_font_serve"/>
+                <span/>
+            </label>
        </div>
-        <p>Adding a font requires a reload of the page. This will save all your changes.</p>
    </div>
    <t t-name="website.delete_google_font_btn">
+        <i t-if="!local" role="button" class="text-info ms-2 fa fa-cloud" title="This font is hosted and served to your visitors by Google servers"/>
        <t t-set="delete_font_title">Delete this font</t>
        <i role="button"
           class="text-danger ms-2 fa fa-trash-o o_we_delete_google_font_btn"
           t-att-aria-label="delete_font_title"
           t-att-title="delete_font_title"
+           t-att-data-local-font="local"
           t-att-data-font-index="index"/>
    </t>
    <t t-name="website.add_google_font_btn">
@@ -120,6 +120,9 @@
            </t>
        </t>
        <link t-if="request and website and website.is_public_user()" rel="canonical" t-att-href="website._get_canonical_url(canonical_params=canonical_params)"/>
+        <!-- TODO: Once we have style in DB, add this preconnect only if a
+        google font is actually used. Note that if no font is used, the
+        preconnect is actually not connecting to the google servers. -->
        <link rel="preconnect" href="https://fonts.gstatic.com/" crossorigin=""/>
    </xpath>