Fuck Symantec
Customer is running two Terminal Servers on Windows 2000 Server. 32bit. 4GB of RAM.
Recently upgraded to Symantec Endpoint Protection 11, around 1 Month ago. A week ago, the customer complained that one of the Terminal Servers crashed constantly, requiring a reboot to recover.
Quick investigation showed that the machine was running out of paged pool.
Event ID 2020
Event Type: Error
Event Source: Srv
Event Category: None
Event ID: 2020
Description:
The server was unable to allocate from the system paged pool because the pool was empty.
I’m not proficient with Terminal Servers or Windows 2000, but debugging this issue was mostly similar to what you do when debugging pool issues on Windows Server 2003. First you need to enable Pool Tagging, which is enabled by default on Windows Server 2003 but not on Windows 2000. KB177415 explains how.
After that, install the Windows 2000 Support Tools, and run poolmon /p /p /b.
In my case, the output looked like this:
The limit for Windows 2000 Terminal Servers is 160 MB. As you can see, the machine here is idle and without any users on it. And we’re already at 132MB utilisation.
There are two culprits: “CM” and “SavE”. The Pooltag “SavE” is the Symantec Endpoint Protection Virus Scanner Driver. It clocks in at 50MB. The other Pooltag “CM” stands for “Configuration Manager”, and is the registry. It is 67MB big.
This is not normal - the other Terminal Server, the CM tag is a lot smaller, only 35MB. The “SavE” tag is still 50MB. This explains why the other TS does not have the same problems as this one. But we don’t know why one registry is so much bigger than the other.
This can be found out by using the dureg.exe tool, which can help us resolve the issue.
As you can see from the picture above, the enlarged registry is also caused by Symantec.
C:\Programme\Resource Kit>dureg /lm “SOFTWARE\Symantec\Symantec Endpoint Protection\AV\Quarantine”
Size of HKEY_LOCAL_MACHINE\SOFTWARE\Symantec\Symantec Endpoint Protection\AV\Quarantine: 26111494
Clocks in at 26MB. The Quarantine key contained around 20′000 subkeys, each with a simple number below. Each was about the same .doc file.
After deleting the Quarantine key, the CM pooltag went down from 67MB to 35MB - just like the other TS.
The next step was obvious. Remove Symantec Endpoint Protection which something that doesn’t suck as bad: McAfee AntiVirus Enterprise. I downloaded an Evaluation Version, and installed it.
And the results were obvious:
Do you see the pooltag “NAI0″ somewhere in this list? I don’t. It’s there, but somewhere around Page 400, and surely not eating away 50MB of my paged pool.
So if you have problems with your machines running out of paged pool, frequently showing Event 2020 with Source Srv, check the registry size and replace Symantec Endpoint Protection with something that doesn’t suck that much.
Stephen Jakes:
First, my systemworks failed by constantly periodically re-requesting activation keys in order to function at all…over and over for over a year. Each time (4 or 5) I dealt with their support, the technician assured me the problem was fixed for good. Meanwhile, my system became infected with the TR/TRAPS.Gen Trojan, which ground all system processes to a near halt. While on hold on the phone with Symantec/Norton for literally four hours, I managed to run Avira’s free antivirus, at my computer was at near standstill speed. On the phone, finally, they first they said their web-support-sites were down globally. Later, they claimed that some Trojan’s render web access, exclusively to antivirus software websites, impossible. Ultimately, I relinquished control of my computer to the Symantec technician via the net. He mosied around for awhile, and WITHOUT MY CONSENT closes the window with my over two hours of Avira progress! Avira had already discovered 4 instances of TR/ATRAPS.Gen and would have quarenteened them. The VERY NEXT thing he says to me is that he can transfer me their virus technician, who is the only one who can delete this sort of problem, for $140.00 base fee for the first hour! His salesmanship consists of threatening me that my “financial and password information” is likely compromised (this ranges from unlikely to impossible according to a software engineer I know, as well as Avira’s information). Enraged, I demand to speak with a supervisor, who turns out to be clearly skilled in the “smoke ‘n mirrors despite the obvious facts” routine. He blames his software’s fifth or sixth failure on me by basically patronizing me. He has an air of superiority, essentially saying that the issue is too complex for my feeble understanding. He refuses to look up the event numbers I provide of past calls/chats related to this same failure of the software. He refuses to resume the active-web-control session I just had with his operator, to support his claims…however, he does admit an event-sequence-log does exist on my machine. To summarize, I am absolutely certain that with the active control feature of the internet, they deliberately sabotaged Avira from fixing the problem and then offered to fix the same problem for a huge fee. Upon reboot, indeed, the Avira fixed the problem, although it took an additional 3 hours to run it’s scan over again. DO NOT EVER DO BUSINESS WITH THESE CALCULATED SYMANTEC EXTORSIONIST’S! Besides, Symantecs technicians all sound outsourced to India, so why should they care or have integrity? Symantec appears multinational enough to UNDERSTAND that they can get away with a lot because their callers are consistently in a desperate position.
9. January, 2009, 12:45