How to use the webencodings.encode function in webencodings

To help you get started, we’ve selected a few webencodings examples, based on popular ways it is used in public projects.

Secure your code as it's written. Use Snyk Code to scan source code in minutes - no build needed - and fix issues immediately.

github webrecorder / pywb / pywb / rewrite / rewrite_content.py View on Github external
if text_type == 'html':
            head_insert_str = ''
            charset = rewritten_headers.charset

            # if no charset set, attempt to extract from first 1024
            if not rewritten_headers.charset:
                first_buff = stream.read(1024)
                charset = self._extract_html_charset(first_buff,
                                                     status_headers)

            if head_insert_func and not wb_url.is_url_rewrite_only:
                head_insert_orig = head_insert_func(rule, cdx)

                if charset:
                    try:
                        head_insert_str = webencodings.encode(head_insert_orig, charset)
                    except:
                        pass

                if not head_insert_str:
                    charset = 'utf-8'
                    head_insert_str = head_insert_orig.encode(charset)

                head_insert_buf = head_insert_str
                #head_insert_str = to_native_str(head_insert_str)
                head_insert_str = head_insert_str.decode('iso-8859-1')


            if wb_url.is_banner_only:
                gen = self._head_insert_only_gen(head_insert_buf,
                                                 stream,
                                                 first_buff)
github webrecorder / pywb / pywb / rewrite / content_rewriter.py View on Github external
def get_head_insert(self, rwinfo, rule, head_insert_func, cdx):
        head_insert_str = ''

        # if no charset set, attempt to extract from first 1024
        if not rwinfo.charset:
            first_buff = rwinfo.read_and_keep(1024)
            rwinfo.charset = self.extract_html_charset(first_buff)

        if head_insert_func:
            head_insert_orig = head_insert_func(rule, cdx)

            if rwinfo.charset:
                try:
                    head_insert_str = webencodings.encode(head_insert_orig, rwinfo.charset)
                except:
                    pass

            # no charset detected, encode banner as ascii html entities
            if not head_insert_str:
                head_insert_str = head_insert_orig.encode('ascii', 'xmlcharrefreplace')

            head_insert_str = head_insert_str.decode('iso-8859-1')

        return head_insert_str

webencodings

Character encoding aliases for legacy web content

BSD-2-Clause
Latest version published 7 years ago

Package Health Score

71 / 100
Full package analysis

Similar packages