Problems With Norton Internet Security 2006 and Compression

Author: Chris Martin (chris[at]ecube.co.uk) 29/Jan/2006 Version 001.

Updated: 25/Feb/2006 Version 002 - Problem Solved?

1. Introduction

A number of customers have reported some problems using Norton Internet Security 2006 with websites, including

Garbled or Screambled Content
Blank Pages
Unexpected "Download" prompts
Errors or garbled PDF Downloads

We have investigated these problems, and isolated a problem with the way that Norton Internet Security handles Compressed HTML Downloads. We are attempting to contact Symantec to report these issues, which appear to be faults in Norton Internet Security itself.

Update: Further investigation has revealed an issue in the way that NIS handles incorrectly structured compressed data streams. We have updated our software to serve the correct compressed data and this problem now appears resolved.

2. Diagnosis

To confirm that Norton Internet Security is responsible for any problems

Repeat the problem with Norton Internet Security enabled.
Disable Norton Internet Security (e.g. by Right Clicking on the task bar icon and selecting Disable Norton Internet Security.
Restart your browser and attempt to access the website again.
Re-enable Norton Internet Security.

If disabling Norton Internet Security fixes the problem, it is likely that it is the cause.

3. Work-Arounds

There are a number of work-arounds which may be used pending a fix to the Norton software

3.1 Disable or Remove Norton Internet Security

Disabling Norton Internet Security when accessing the affected websites. If you are following this option, please make sure you have other security measures (a firewall in particular) in place.

3.2 Configure your Browser to use HTTP 1.0

By configuring your browser to only use HTTP 1.0, the browser will not request compressed files, and the fault in Norton Internet Security will not be triggered. To do this in Microsoft Internet Explorer:

Select the Tools menu, then Internet Options....
Click on the Advanced tab.
Scroll down to HTTP 1.1 settings
Clear both boxes starting Use HTTP 1.1 in this section
Click on OK
Restart the browser.

Note that disabling HTTP 1.1 will increase the time taken to display all websites viewed with that browser.

3.3 Use Privacy Control to diable compression

The Privacy Control feature of Norton Internet Security may be used to "hide" your browser type from the website; thus disabling compression and so avoiding the Norton fault. To do this

Open Norton Internet Security
Double-click on Privacy Control to bring up the privacy controls.
Click on Advanced
Add the website to the list on the left by clicking on Add Site in the bottom left of the box.
Enter the name of the website (eg www.investorsintelligence.com, or www.fullermoney.com, or www.stockcube.com as appropriate) and click on OK.
Check the website name is selected in the left hand column.
In the "Information about your browser" box, clear the Use Default Settings box, and then click on Block.
Click on "OK", and then "OK" again.
If Privacy Control is not turned on, turn it on now.

Appendix One: Technical Details

This section for developers.

We welcome any comments or help from other developers who have encountered this problem. please email us on chris[at]ecube.co.uk.

Port80 Software have reported similar issues with Norton Internet Security (see here and here).

We have examined the traffic between the server and Norton, and the browser, and are able to confirm the details in the Port80 note; that Norton is attempting to un-compress the HTTP data, and pass it back to the browser as Chunked un-compressed data. We note the following

When it fails, the "uncompressed" data passed from Norton is corrupt binary data - probably the compressed data stream from the server, but without the correct Content-Encoding header present.
On very short items (10-20 characters) it is correct but truncated.
The corruption does not appear to be affected by the HTTP headers.
The compression parameters for the "deflate" compression do make a difference, though not in any monotonic relationship; in one example, compression levels 1-4 were corrupt, 5 -7 were OK, and 8-9 were corrupt again.
The other "deflate" parameters (memory, stragtegy, etc - we use zlib) also make a similar (non-monotonic) difference.
Using the latest zlib made no difference.
The content of the page also can make a difference to whether the fault is triggered.
Compressing at level 0 (ie passing through the blocks in plain text) always works correctly.
We have yet to see the problem with Apache's mod-gzip, which we note uses it's own compressor code.

From this we guess that there is a fault in the Gzip decompressor contained within Norton Internet Security, which is not able to handle some of the data compressed with zlib. We guess that Apache's mod-gzip deflate implementation does not trigger this fault (if it did the fault would be reported more widely.)

It would be interesting to compress identical data with zlib and mod_zlib and compare the outputs.

We can repeat the problem with Win32 version of curl (http://curl.haxx.se) running on the machine with Norton installed, with the following command:

curl -A "MSIE 6" --compress --trace - www.stockcube.com/x/default.html

The responses from this command with Norton enabled and disabled are as follows

Enabled
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Date: Sun, 29 Jan 2006 03:10:49 GMT
Server: Apache/1 .3.33 (Unix)
Cache-control: private,max-age=0
---------------- : ----
Expires: 0
Pragma: No-cache
 Vary: Accept-Encoding
Set-Cookie: TEST COOKIE=-16; path =/
--------------: ----
Content-Type: text/html

BAD.............
Y.R.....T...>..f
...

Disabled
HTTP/1.1 200 OK.
Date: Sun, 29 Jan 2006 03:19:50 GMT
Server: Apache/1 .3.33 (Unix)
Cache-control: private,max-age=0
Content-Encoding: gzip
Expires: 0
Pragma: No-cache
Vary: Accept-Encoding
Content-Length: 2989
Content-Type: text/html

...........Y.R..
...T...>..f.7.%!
...

Note that with Norton enabled; the Transfer-Encoding is now "Chunked", and the Content-Length and Content-Encoding headers have been dashed-out; correctly as the data is now supplied "Chunked" and un-compressed.

Note also that the data sections are identical after the "Chunk" header; implying here that Norton is passing through compressed data without the "Content-Encoding" header ("BAD" is simply the hex-encoded chunk size).

Update: We have identified a problem in our compressed data stream; the CRC and ISIZE (RFC 1952) were missing. Adding these solves the problems with Norton. It seems that browsers are much more tolerant of this missing data than NIS. We are still attempting to report this problem to Symantec.