How to Clean a Hacked Magento Site

Cesar Anjos - Incident Responder

If your Magento website has been hacked, learn how to appropriately deal with the security incident, fix the hack, and secure your ecommerce website against future breaches.

This webinar will take place on Wednesday, Feb 22nd at 11am PST. Following his presentation, Cesar will take questions from participants. Please complete the form to register.

Porto • Portugal • Home of Sucuri's
Cesar - Incident Responder

Cesar Anjos

Cesar is a Security Analyst who spends his free time researching. One of his main concerns is privacy. As such, you won’t find him on social media.

Questions & Answers

Question #1: What security plugins do you recommend for Magento?

Answer: Most important is a two-factor authentication plugin, most Magento attacks have a starting point on the backend so if that area is fully locked down this simple step will go a long way to keep the website secure. On sites with big stagg it may be also useful to get admin actions log plugin. A good external firewall should have all that’s necessary. There is no way to safely use out-of-date/vulnerable software.

Question #2: If we check our Magento version, it says security patch 8788 does not exist. How can we check if our website is secure?

Answer: The Best way is to rely on magereport.com as they will analyze your site for any missing patch or vulnerable areas that the attacker’s exploit.

Question #3: Where are the most common places to find malware in Magento?

Answer: As most Magento sites are used to steal private information from your clients a quick look at the footer, header, and miscellaneous HTML area through the backend areas such as blocks and design configuration should alone be a big step on finding the malware. It can also be spread through Javascript files that belong to Magento’s Core as well as some of the files that handle the checkout process directly.

Transcription

Cesar

Hello everyone and thank you for joining. My name's Cesar. I'm an incident responder at Sucuri. I'm part of the team that handles the incidents that our clients bring to us. I'm from Portugal which is that little country in the corner of Europe. In this webinar I'll try to help you understand if there may have been any compromise on your website and if any part of that compromise could have been credit card stealers. I'll share with you how credit card stealers operate and how you can look for almost all variations of it and how you can get all that malware cleared from the website.

I will try to present all this information in a way that everyone can understand regardless of technical level. We have an area on our website dedicated to some very useful guides that we write. We have just recently released the Magento Cleaning Guide where the PC Compliance subject is approached so it should definitely be checked out.

Now, what exactly is data stealer malware. Data stealer malware is just a kind of infection that takes the data that your customers use to make purchases on a website and steal it. This kind of malware is the biggest enemy of any Magento website. This is because our private data is usually used on a daily basis and getting that data stolen is very valuable for attackers. The damages that's inflicted can be very severe both for you and your clients so it may be a good idea to get the authorities involved depending on the severity of the breach and how much data may have been stolen.

I strongly recommend that that you consider locking out clients if any breach happen as that data may have not yet been used by attackers. The webinar is divided into a few sections, how to check if there's been any compromise, how to look for the malware, and how to clean it, and what steps can be taken to minimize the risk of a reinfection.

It's important to understand the implications of having a data breach on your website. If there's been any report or confirmation of a breach it's important that it's investigated as soon as possible because any data that is stolen from the website is usually used within the first 12 hours. In some cases the attackers just save the data for later or just try to sell it to someone.

The technical term for the credit card data being sold is called carding. Many probably have already heard about it. Before we get started with the process it's very important to take an immediate backup of everything related to the website and actions taken there, logs, files of the website and the database. All these may be needed either for the investigation or to recover from anything that could have gone wrong or got broken during the cleanup.

Now, let's go ahead and look for signs of compromise on the website. There's a few scenarios where a compromise can be the suspect. For example, if your customers report any strange behavior on the website, being it redirect, severe issues on the checkout, malware being flagged by external scanners, or just the clients complaining that their data was stolen after they bought something on your website. Also if you notice something has changed or shouldn't have been there or you noticed an unauthorized access on the website where a blacklist was imposed this is a clear sign that a compromise probably happened.

Now, the most common indicator of a compromise on Magento websites is just the report from the clients. It's very important that you discuss with them and you get as much information as possible. You need to understand what exactly is being reported and you get as much information as you can about the issue. If the report is about data theft you may need to try to validate that it was actually through your website that it happened and you try to establish a timeframe that will assist on the investigations.

You can ask questions such as, "Was the card used on any other websites? When exactly did the purchase take place?" and so on. You can also take proactive measures, which is for example, you just try to make a purchase yourself on your website. The problem with this is that if you use your own credit card on your own website that might potentially be infected is very risky. I'll get into a more secure way to do this later in a minute.

There's many ways that your checkout can behave badly. There can be redirect, pop ups, strange files being loaded on the checkout, payments ending up on someone else's account, or just the checkout stopped working all of a sudden. You have to ensure that the entire checkout process works properly at all times, both for your safety and your client's safety. Here we have an example of a bad behavior on a checkout page. You can see marks by the red squares that there's a few files loading from a domain called trafficanalyzer.biz. On the right side you can see that there's all the data that was inputed on the checkout form that is being sent directly to that domain.

This is a very clear behavior of how a credit card stealer works and in this case it has to be dealt with as soon as possible. In this case it's one of the most easiest to investigate examples because we have a direct domain name that we can just directly search for on the files of the database and it becomes much easier to find.

Now, if you try to make a purchase on your website you can just make use of a virtual credit card which is a much more secure way to do online shoppings. These kinds of cards are all throughout the world either directed by our banks or any other external services. These kinds of cards have absolutely no relation with your actual identity or bank account. Since they also have a hard spending limit they'll always ... The risk if they get stolen is really low. Even if they manage to steal any amount of money it will be just at the top of what you set.

These kinds of cards are ideal to test on your website for any credit card stealer possibility. You can make a credit card with a limit, let's say, $10. You make a $1 purchase on your website. This leaves a margin of $9 that the attackers can use and from there you confirm that it actually was stolen.

Some of these services that offer the virtual credit cards also allow you to keep track of any transactions that were failed either by lack of funds or any other reason. In these cases you don't even need to leave extra funds there for the attackers because you can just look for failed transactions. You have the card. If any transaction happens that you did not do or it's attempted you know that it's being misused by someone. It's very important to keep in mind here that even if your data is not used by an attacker it does not mean that the data wasn't stolen.

It may have just been that it wasn't just used yet or they just decided not to use it for now. They just skipped it. They have a suspicion that there's something fishy going on there. They suspected you are trying to track them and they may not use it. This is a very rare occurrence.

There are a few tools that you can use to scan your website or just check if it's blacklisted. We have a few examples, such as Sucuri SiteCheck, MageReport, major scan, or virus total. I strongly recommend that you make regular scans with SiteCheck and MageReport because they provide the most insights on your website, especially on Magento and most of the occasions that you may need.

Here we have an example of SiteCheck, which is very handy. It checks for a few blacklists. It scans for malware and it gives you an immediate report of what it detects so it's very good information that you can add directly to your investigation process that will come next. MageReport checks your site for missing patches, any element that may be vulnerable, such as slash downloads. It also checks for certain kinds of malware that are very specific to Magento such as a credit card stealer called Visbot as you can see here on the top right of the image. In this case the site was infected with it so it immediately alerted.

Google has also recently started to flag certain credit card stealers. It's very important that you should get this handled before Google detects it or the traffic to your website will come to a standstill very very fast. This is a very rare occurrence but it's a possibility so you want to avoid it at all costs. In some cases usually it's good to just keep a regular check on VirusTotal which checks several blacklists directly through your website. It's usually live data so it's very good to keep an eye on this.

Here we have an example of Google blacklist which everyone always tries to avoid and in credit card stealing the damage is not just for the traffic or for you; it's because your clients are also severely effected as well. We need to keep a very close eye on these situations. The most common behavior of credit card stealers malware can be divided in roughly two categories. One of which sends the data to external domains and it's usually injected directly onto JavaScript files or as JavaScript code on the site.

It usually works by looking at what the current page is being loaded and it tries to work only when it's the checkout page such as slash five checkout and so on and so on. The same kind of behavior can be found on that malware that also tries to steal login credentials. That kind of malware can just look for slash admin which is the default page and it just tries to act just on that page so that it doesn't either get extra data or something that the attacker isn't interested in.

Because it's usually in JavaScript files this is usually cached by Magento JavaScript merging functionality which is a mechanism that Magento has where it takes a bunch of JavaScript files and just joins everything into a single file to make the page load go much smoother. Instead of having to load five or six files or more it just loads one or two. It becomes much more efficient. This kind of malware also gets sometimes injected as PHP code onto PHP file but this case is much much rarer.

The other kind of malware stores the data directly among your website's files. The data that is stored on a file is usually disguised as an image or a file with no extension within the media directory which is flash media. Most cases. This kind of method makes a file go over three megabytes very very quickly. The attackers usually encrypt the data being stored so it can only be read by them, which makes sense because they just want them to have access to the data and because of this encryption the size of the file grows very rapidly. Usually just one or two lines can easily go over one megabyte.

This kind of malware can only be injected on PHP files because JavaScript cannot take any direct action to any file present on the server because JavaScript runs directly on the visitor's browser. It isn't even handled by the server, so the server just sends it and the browser runs it.

Now there is also a third kind of data stealer malware that just takes the data and emails it directly to the attacker. This can only work as PHP files and is very very easy to discover, because all it takes is if you have email logging enabled on the server you can just ... You either use a monitoring system for the emails or just look at the logs yourself and you will immediately see that there's something there that looks very strange to you and should not be there.

Now it's time to find the malware. There's no need to go blind into the investigation. It's best if we start by using the data that we already have. We have the behavior that was plotted or reported. We have a timeframe that we tried to establish and any other information so that we can know what we are looking for. Such as what was reported by the scanners and so on and so on. From there we then look for other kinds of infections that may be present on the website such as back doors or anything else that the attacker may have tampered with.

The best method to look for malware is just to look at what files are different from what they're supposed to do. If you manage to establish a possible timeframe for when the compromise started you can try to look for files that were changed closer to that date. You can also give a try to Amnesty's free modified core files reporting tool which just compares your Magento installation with an integral version of it.

One issue with this tool is that it doesn't check all of the core files of Magento. It only checks the very most essential files for the installation to work. There's many files that are not compared and you cannot trust this 100% but as a quick investigation tool it works great because it also runs as PHP so it's very easy for anyone to run it.

Here we have some information how to use the dif command through command line interface where you can compare the current installation of Magento with a freshly downloaded one. It's important that it's the same version of it. Either 1.5 or whatever. It reports back the differences that it spotted. From there we can greatly narrow down the investigation scope because we don't even need to look at files that are as they're supposed to be. We can just look at any differences.

A note here is that any modules that you may have installed will have to be compared separately because it will not come with that installation. It's a very good idea to keep all the original versions that you installed stored somewhere secure. If you update you try to get an archived version of that update and store it somewhere as well. There's also many programs that you can use if you are not familiar with command line interface that can allow you to compare two folders.

An example we have for Windows, for example is DiffMerge. There's various that work on various operating systems. A solution is always there. You can also look for files that were modified some days ago. Either 10, 20, 30, 40 days, whatever. This can only be reliably trusted if you manage to establish a good timeframe for the compromise because otherwise you will just get unnecessary noise that may just send you in circles because there either that may not be of interest to anything for the investigation.

As I previously mentioned files that hold stolen data on your server can easily grow over three megabytes. It's a very good idea to look for big files. It's very rare that any actual web file will be over three megabytes. Here we have a command that just looks for files that are bigger than three megabytes. The result of this command will be mostly a few images with very high resolution and some archives. You should be able to distinguish between both and you know what you added or should be there and what is not supposed to be there. You can focus the investigation on those files as well.

At Sucuri we have a backup service that takes backups of your website every day and if each backup is done, you receive an email telling you how many files were added, how many files were removed and how many files were modified. This works great as an integrity records keeper as you can also see which exact files were effected. It makes a very good and cheap integrity monitoring solution as well as providing secure backups of your website as they are stored away from your server keeping them safe from anything that may happen or infection. This is an add-on service that we provide outside of our normal plans.

Many of Magento's initial compromise starts with a creation of an admin user on the site. It's very important that you regularly keep an eye on this and remove any user that should not be there. Here we have two examples of users that were added by an attacker. They usually use ... Many times the email user at example.com or example like the example.com or anything else. The important part here is if you don't recognize and you know it shouldn't be there you have to remove it immediately and look for any compromise.

Here we have the most common functions used by attackers when the malware's objective is to steal data from your clients through your website. If you search on your files or database for these functions you should be able to find at least 98% of the variations of this kind of malware. Unfortunately you may need to have some PHP knowledge to make sense of some parts but any developer should be able to assist you with this.

You can search for any of these functions either through command line interface or any program that allows you to search for text on files. These two examples of data stealers that were present on the one page control .PHP file. This is the file that directly handles the checkout process on Magento. On the left you see the code using file get contents to send the data that was inputted on the form, on the checkout page to the Excel domain directly and from there the attacker either stores it, uses it, or may even try to sell it.

On the right side you can see the code that's using file put contents and file append function to store the stolen data on an image named cancel on the effected site server. In this case it's a very good example of what I previously mentioned where you could just search for big files and it will immediately flag this one. It could probably be few hundred megabytes by the time you detected it so it would probably be very easy to spot. These two infections are present on the exact same file so the data was possibly being stolen by two different attackers which makes matters much worse because there will be two people using it.

Here we have some of the most common functions used by back doors within Magento. There's also possibly some premium models that make use of some of these so it's important that you check the integrity of the modules before searching for back doors so that you can rule them out and not spend time searching for something that's unnecessary.

Here we have some examples of common back doors. If you find any code that looks just like them they are most likely back doors. Most of these back doors just take the code that the attackers sent to them and execute it. It can be from injecting some other malware to deleting anything that they want and just one or two lines of code that can achieve a lot. It's very important that you keep a big eye on this and remove it as soon as it's possible.

In any investigation it's important that you look at your site's filter and data areas. Either through the database or directly through Magento's backend because they are a big target for attackers. It's something that only requires access to the backend to add and since it loads everywhere on the website it allows JavaScript to be placed. It makes a high value target for attackers. Anything malicious added on these areas is usually very easy to recognize as it was not added by you or your team and it's something that doesn't require any technical knowledge to check.

This is an example of an infected footer links block where a credit card stealer was injected. It's encoded, yes, but it's easy to recognize that it's something that shouldn't be there and no web master would probably add something like this. In these two images we can see the design configuration area of the store. Both miscellaneous scripts and miscellaneous HTML were injected by another data stealer malware. In this specific example we can see that the code is looking for the checkout page where it then executes an external JavaScript file.

In this case it looks specific for one page, checkout, one step, or fire checkout. If the current URL matches that it can run circles. If you have trouble finding the malware, cleaning it, or any part of the process, or just want it to be handled by an expert just seek out help from a professional. It's better to be safe than sorry. Everyday that there's malware present on your website there's damage being done to you, your reputation, and your clients.

Security is available 24/7, 365 days a year, so if you need help you can just come to us.

Now, it's time to go ahead and clean up the website. Before the cleanup can be started you need to be sure that the backup is fully done and stored in a safe location outside your server. I recommend that you put our website onto maintenance mode at least while you are cleaning it to avoid further damage. As I previously mentioned, the back doors are a big pain and if you are cleaning the website but there are still back doors there you may be in the middle of the cleanup and the attackers may be already attacking your website and reinfecting everything again. It's very important that you try to put your website down or in maintenance mode before you clean it.

If you haven't any technical knowledge or technical ... If you have a technical knowledge or even a technical team you can even put the website on a temporary location such as a staging environment and you attempt to clean up through there before moving it back to the live environment.

If there's a possibility of high-volume data being stolen you should consider reaching out to the authorities. They will immediately ask you for the files of your website, your database, and any logs you can grab. You should already have all of this stored in a safe location by now.

To fix the hacked files you start by looking for any malware indication you may have previously obtained such as results from an external scanner or the reports from the client. You then inspect the modified files that you found and there you can directly search for the malware related functions we previously saw. If you find any suspicious code you can use some tools to help make better sense of that code. For alternative you can simply restore their files to their known good version by just replacing them. For example if there's an effected module you can simply replace those files with a freshly downloaded version of it. It's done, you can move onto the next.

In the end you just have to test to the site to ensure that everything still works fine especially the checkout which is probably the most important part. If any of the modifications is something that you yourself changed or your team then you have to keep them in mind because they will be reported by the previous steps and you need to keep in mind that you yourself changed that or your team.

Not all that is different is malicious. It may be just ... It can be called a false positive. When in doubt it's best to double check. Even Google can help find the answer to any suspicious code that you may found or any doubt that you may have.

As I previously mentioned some modules in Magento come encoded to protect the codes. They can easily look suspicious and that's why very important that they are compared with their original version separately. In case there's some extra code on them you just want to know where it does. You can use excellent tools such as DD-Code and PHP or JSBeautifier which is just for JavaScript code. If you don't have any knowledge regarding programming or code you can just consult with your developer. They will tell you exactly what the code is doing.

Here we have some examples on how you can easily take an encoded code and use it on the external tools to have them decode it. In this case here we have DD-Code. You just go there, put the code, hit the button and it tries to decode it and presents the results. It's very very simple. Here we have UnPHP, also the same way to operate but it's good to keep in mind that when one of them can't decode it the other may be able to. It's always good to keep both.

Database infections are very rare to contain spam within Magento. The attackers prefer to keep the spam out and usually they like to use only credit card stealers because it makes the webmaster have a harder time realize that anything actually happened on the website because they leave any trace of anything else the webmaster may not realize it. The attackers usually focus the damage on the core config data table. More specifically the records that relate to the header, footer, and miscellaneous area.

This also means that it can be easily cleaned by just removing the code through the actual website's backend. Just login, check those areas manually, and you immediately see the code there. That's usually all that they do.

Now to wrap it up we have some finishing touches to do. We need to reset the credentials of any users with elevated access on the website. Change all credentials regarding STPOs or SFTP as well as if it's used. You to need review the current elevated access users and check if they actually need that kind of access. Concept of list privilege applies here. You should only give as much access as that person needs and when they no longer need it you take it back.

It's also very important that you clear all of the caches on the website. It's very very important. I previously mentioned that the good part of credit card stealers get easily cached by Magento. You may cleaned up the website and believe it's clean but the malware may still be in the cache and from there the process can restart all over again.

And if any blacklist was imposed on the website that's going to need to be handled as well. Now that we have analyzed all the files and the most important database locations and we cleaned everything we have, we have to ensure that we take a few precautions to minimize the chances of reinfection as much as possible. It's important to have all security badges applied on the site to reduce the number of risks. You need to have a good backup system in place in case anything bad happens.

All computers with access to the backend have to be scanned for malware because malware can easily go from computer to the website as it can go from the website to computers. A good practice is also to change the URL of your website's backend, the slash admin, change it to something else that the attackers may not be able to figure out. You can also make regular purchases on your website but of course using the virtual credit cards that I previously mentioned to ensure that everything stays safe and that there's low risk of any data being leaked from your website.

Here we have a few good examples of what can be added to your website to get it better secured. We have firewalls, two-factor authentication systems, those are the most important things. In this case we have Sucuri firewall Mage Firewall Security, and then two two factor authentications. The last one, it's very good for especially big websites where they have a big team where it logs every action that your admins are taking. If any of those accounts get compromised an attacker manages to login you can easily see what was done and so on. From there it's much easier to mitigate the damage that was done.

If the attackers got in it's possible that they may return so it's important that all accesses are constantly monitored to ensure that there is no unauthorized access. It's a good idea to take a look at the list of users on the site every once in awhile. We should also plan with your team what they should do in case anything happens. What to do, who to contact, and so on.

If you have a very high traffic website this is very very important as the damage of any compromise is probably going to be very big for your business. Here we can also employ the concept of fire drills. Pretend that something happened on the website and see how your team responds. From there you can also check that your backup systems also work. You can just try to restore its Backup location, see if the restore goes well, everything is restored properly and so on. The objective here is to be preventive as well as reactive. You try to prevent anything from happening and you plan for it in case it does happen in the end.

A good firewall should take most of the burden required to keep your website secure. Our firewall has all the features to keep your website safe even for the mostly backend web master. He we have just a brief overview of the features that it has. It's also good idea to check out. I guess that's it. Thank you very much everyone.

Questions & Answers

Question #1:What security plugins do you recommend for Magento?

Answer: Most important is a two-factor authentication plugin, most magento attacks have a starting point on the backend so if that area is fully locked down this simple step will go a long way to keep the website secure. On sites with big stagg it may be also useful to get admin actions log plugin. A good external firewall should have all that’s necessary.

Question #2: If we check our Magento version, it says security patch 8788 does not exist. How can we check if our website is secure?

Answer: Best way is to rely on magereport.com as they will analyze your site for any missing patch or vulnerable areas that the attackers exploit.

Question #3:: Where are the most common places to find malware in Magento?

Answer:As most magento sites are used to steal private information from your clients a quick look at the footer, header, and miscellaneous HTML area through the backend areas such as blocks and design configuration should that alone be a big step on finding the malware. It can also be spread through javascript files that belong to magento’s Core as well as some of the files that handle the checkout process directly.

Question #4: If I cannot patch my Magento website in time, how does the firewall help?

Answer: The firewall has virtual-patching that automatically protects your site against all the vulnerabilities that a patch would fix, so even if you don’t apply the patches your site will already be protected. Here’s an overview as well as more info:

Question #5:Is changing the checkout name a reasonable prophylactic measure? If attackers are looking for “checkout” or “firecheckout”, would naming the page something else like “/makeyourpurchase/” make it so the malware might not work?

Answer: It may work, but only to a very limited degree. It’s just very easy for the attackers to find out what you changed it to so the attackers can just adjust the malware.

Question #6: Does PCI offer any guidance for incidents?

Answer: There’s several good information that can be obtained from pcisecuritystandards.org, such as Responding to a Data Breach - A How-to Guide for Incident Management

Sucuri also has a few great articles worth checking out on PCI subject:

Question #7: How often is Magento targeted by hackers? Is there a report about it?

Answer: Magento is not a big target when compared with Wordpress for example but that may be mainly due to the fact that wordpress holds a larger percentage of websites using that CMS. Although Magento is a much more attractive target for attacks due to the fact that large volumes of valuable data make it through Magento websites everyday.

Hacked Website Trend Reports

Question #8: Some problems come from badly coded themes or extensions. Can you give some suggestions on checking for code that would allow vulnerabilities?

Answer: Unfortunately this matter can be very subjective, some kinds of codes can be easily recognized as vulnerable even by a developer, but in most cases you need an actual vulnerability analysis on the code which is very expensive and not worth it. It’s usually better to go straight to having a firewall that protects from any vulnerabilities that may be present.

Question #9: How can Sucuri WAF help to prevent malware?

Answer: See question #4

Question #10: Cesar, Willem de Groot recently wrote about a relatively new type of self-healing malware out in the wild that appears to be infecting the cms functionality of the database (skirting around the core_config_data table stuff that used to be more common) using triggers to re-install itself even if you manage to remove the infected javascript code...are you guys seeing this as well?

Answer: It shouldn’t, you just have to ensure that you whitelist the firewall’s IP so that the firewall doesn’t get blocked.

Question #11: If I use the Sucuri firewall does that affect an interface I have with another service that whitelists IPs?

Answer: It shouldn’t, you just have to ensure that you whitelist the firewall’s IP so that the firewall doesn’t get blocked.

Question #12:Isn’t it quite easy for a hacker to “fake” the file timestamp?

Answer: Yes it is, that’s why the timestamp alone is not enough for a proper investigation, timestamp does help in some cases as the attackers sometimes fix the timestamp on the file to be the same on the other files but the one on the folder above shows the modification..

Question #13: Hi, maybe a stupid question, but how come the Sucuri scan recognizes my Magento website as a WordPress website?

Answer: Oh really? Maybe you have some WordPress files there, feel free to reach out to us so we can check.

Question #14: Is there something similar for WordPress? For site protection/server protection?

Answer:Most effective way is data correlation between behaviour that is happening, the reports received, blacklists and information that comes from the logs. Although checking the logs can require some technical knowledge.

Question #15: Most local, state and federal jurisdictions (as well as credit card processing gateway providers) have very strict regulations about how hardware & software systems and supporting network infrastructure are preserved for the criminal and civil investigations that are required by law and your contracts with merchant gateway and clearing providers. Do you have any recommendations on how to specifically use the tools or processes you discussed here so that you don’t accidentally wind up committing a felony by destroying evidence by cleaning up a site?

Answer:Usually the authorities will only require the files of the site, databases and all the logs from the server, but in some cases usually very severe ones they may require that absolutely nothing is touched, in such cases if you want to cooperate but still want the site online its best if you check with your services provider about making a complete cloning of your server onto another one, so you can bring the other one back online and lock down the first one. Alternatively you can also just point your domain to a different server and restore some backup you may have of your website there to get it working or just put a maintenance page there, this will keep all evidences intact while prevent the attacker from going back into it and tampering with the evidences.

For For further info on this just check this PDF by the PCI Security & Standards Council on "Evidence Preservation"