Update 9/Jun/2015 – Thanks to Josh Davis for the feedback, I’ve added a note about making sure to include both drives (if EDB and LOG files are separated).
Update 21/Oct/2013 – This article suggests that you cannot sustain downtime or interruption for your users while battling with deleting log files or restoring your working backup solution. If you can sustain a downtime (should be around minutes or so) the easiest method will be to enable Circular Logging on your database / storage group – see more here – http://technet.microsoft.com/en-us/library/bb331958%28v=exchg.141%29.aspx#UTL
Update 01/May/2013 – The exchange team has written a script which helps troubleshoot and identity issues with Backups etc.. The script use the DiskShadow utility as well ! check it out @ http://blogs.technet.com/b/exchange/archive/2013/04/29/troubleshoot-your-exchange-2010-database-backup-functionality-with-vsstester-script.aspx
Hi Again !
I often get calls and questions regarding backups and Exchange Server, since ever this issue is not always working as required or as you would expect, but that’s off-topic 🙂
One of the most common stories is that without a working Exchange Server backup when you perform massive mailbox moves, transaction logs will get piled and fill up the volume or disk that they reside in. and then panic starts, “hey my databases were dismounted…” then of course the administrator realizes that the space on the log drive or volume has indeed ran out and now he needs to figure out what to delete.. and here’s where this post comes in…
So how can you delete or purge Exchange server logs without any risk ? well, in simple – you cannot, because the whole idea of restoring an Exchange or for this matter any transactional database requires you to have a first – “full” backup of the database itself and all transaction logs that were generated since the the date of the database creation date, or the last “successful” “full backup”.
Now here’s a nice method to “fake” a “full backup” or an on-demand transaction logs purge when you see you will be soon out of space, using the Exchange VSS writers and the diskshadow utility (available with Server 2008 or 2008 R2) . This procedure also “proves” that a VSS backup for your Exchange Server will work fine.
note: This method was tested on an Exchange server with Locally Attached Disks, not storage attached LUNs.
Use this method on on your risk. You should preform a “Full Backup” right after this process is done.
This example will show you how to purge the logs for a database that is located on Drive D, the log files of the databases are also located in Drive D. we will “fake backup” drive D and this will trigger the logs to be purged.
Note: If you have separated your log files and database file in different drives, or you want to include additional databases in the “backup” you must include the additional drives in the process, so in the example below, you will “Add volume e:” after “Add volume drive d:” and so on…
- Open Command prompt
- Launch Diskshadow
- Add volume d:
- (optional, add one line for each additional drive to include) Add volume X:
- Begin Backup
- Create
- End Backup
- At this step you should notice the following events in the application log indicating that the backup was indeed successful and logs will now be deleted.
Here’s some screenshots from the process:
The Diskshadow example screenshot.
ESE – Event ID 2005 – Starting a Full Shadow Copy Backup
MSexchangeIS – Exchange VSS Writer preparation.
ESE Event ID 224 – Logs are now purged 🙂
MSExchangeIS Event ID 9780 – Backup is now complete.
side note: although this example was tested against Exchange 2010, it should work just as fine with Exchange 2013 & 2007.
Hope this helps you !
ilantz
Great!! thx!!
Another way to resolve this problem is to set the file properties on the older log files to ‘compress’. That usually get’s you enough space to mount the database and run a full backup.
Indeed a great method to get you going,
thanks for sharing!
What if the mailboxdb is on the C: drive? Is there any danger in using these commands on c:.
Not really. Same effect
Brilliant Idea – Was trying to move all mailboxes to another storage space, but about 50% the way though the Logs filled the remaining space. This freed up enough to get the store back online and finish the job – saved me having to move 1TB to another space just to finish the copy.. Thx (3 years later)
I believe it’s not supported to compress transaction logfiles.
Do you have a reference for that? I’ve used this method a couple times to get me out of a bind. I’ll stop if it is unsupported.
Found some “official” links:
http://technet.microsoft.com/en-us/library/aa996103.aspx – Exchange database files are being written to a compressed folder
http://blogs.technet.com/b/exchange/archive/2004/05/17/133770.aspx – More on Exchange logs
Seems to be “supported” excluding the current and res log files..
thanks……………………….
You’re very welcome! Drop by more often for more 🙂
Hi,
thousand times THANK YOU. My EXCH2010 was nearly using 1TB of disk space for logs. The consistency check for the backup has taken more than 20 hours before the backup has started.
My backup wasnt working the last two weeks and so the logs become more and more………..
Great workaround!
Thank you
Best Regards from Germany
You are very welcome!
Glad you found this useful!
Ilantz
Your a champion. This saved me a lot of headache..
You’re welcome! Glad to hear this worked out for you.
Will this fix work on a DAG?
Yes. This should work on a DAG just like any other mailbox server.
very nice. thank you. It works perfectly 🙂
I found out another method although maybe not recommended but works, stop the information store, just delete the older transaction logs (longer than a month or so) get some free space.restart the exchange information store. run a backup. that will clear all logs and reclaim the space.
Well… I would strongly advice against this. If any of you out there choose this path – by all means – move first the files away, and only after a successful mount of the database delete them.
Make sure you understand that by deleting log files without proper backup (also following this post) you lose your point in time restore option.
Ilantz
Pure Genius! Thanks very much, this works beautifully
thank you very much for the great solution.
i have a question:
last week my exchange server 2010 was unexpectedly restart,
since then i am getting A bad page link (error -338) and my exchange database logs folder is grow up
every day like 30 gb per day/
whay can i do?
esutil?
or the solution you gave?
Kobi,
That event is bad news sadly…
You can do one of the following:
1 – eseutil /p , then /d and hope for the best
2 – restore the last good full backup and roll forward the logs until the current date
Either way, you must take action immediately.
Ilan
thank you for the fast reply/
the strange thing is that everything is working my dadabase is online
what do you reccomended?>
Another way to go is to create a new database and just move mailboxes to it.
Pray that all will move 🙂
Good luck!
i have a previous backup/
which logs i need to rol forward
thankyou
Well kobi, rolling forward the transaction logs can result in data loss, i would highly recommend you to ask for help if you didn’t had any previous experience with that situation.
ilan
I’m have the same issue: accumulating log files in Exchange 2010.
But instead of deleting the said log files, I transfer them to our NAS while the database is still mounted…
This will be my first attempt to move the old log files to NAS while the database is still mounted.
Hope it works. =(
moving logs around will merely solve the space issue.. not the cause of course.
I’d recommend you should troubleshoot the issue and solve the source..
the strange thing is that my database is working without problems\.
just my exchange database folder get big with logs every day like 30 gb/
what do you think about it?
well, you need to determine the root cause of it..
you can try my post and see if it works out for you.
your initial comment talked about an ESE error which I think you must address first.
if i wont backup a vm backup the server i am getting a clean logs:num of pages are updated without any error. i think i will try first your post and then i will backup the database.
I heard:
in the Exchange Database properties select ‘Enable circular logging’.
Dismount the database
Remount the database and this initilises a database cleanup.
Like I say ‘I heard’. I’ve not tested this…
Terry, valid point, thus Circular Logging will require you to dismount the databases and that will cause service disruption.
I’ve just realized I didn’t wrote about that in the post. I’ve added a comment about Circular Logging.
Thanks for your feedback!
ilantz
sorry, can provide much clear step ?
currently the path to store DB has out of space and automatically dismount, due to less of resource the HDD cannot replace or extend
thus, can I use this method, like
i. disable the exchange service
ii. manually remove all *.log file
iii. run diskshadow as you provide
after that exchange 2010 can mount up the db since it has been treat as a “full backup” conduct ?
how about e00.chk file ?
thanks
Lawes
Hi Lawes,
Hope you’ve already managed, but here’s my input..
This method required that the database will be mounted, if it’s already dismounted, then what I’ll suggest is to sort the log folder by date descending, then delete or move the files just enough to get by.
Leave the rest of the files intact.
Hope this worked out for you.
Ilantz
Hi Ilantz – You master!
You just saved me from a potentially very nasty situation. Started getting 452 4.3.1 Insufficient system resources. Checked mailbox log store – Whoa, 139000 transaction logs and down to 12gb of free space. (Wouldn’t have had a hope in hell trying to repair DB from ‘Dirty Shutdown’ if the store had crashed). Further investigation showed MS VSS backup schedule had been deleted.(Don’t ask – too many admins!)
Ran above – What a pleasure. (Less than 5 min of work). All good. Now to go and kick some techies a**s.
Many thanks
Hey ! I’m glad my post helped you out. Thanks for your feedback.
Will this work on Exchange 2003? Also, the log files are on a different drive, so do you add the volume the logs are on?
Hi,
For server 2003 you’ll need to use the VSHADOW command, I believe it will allow you to perform the same procedure, although you’ll need to check the syntax for it before.
Here a link to the documentation and it has the link for the download of the SDK version that holds the VSHADOW command.
http://msdn.microsoft.com/en-us/library/windows/desktop/bb968832(v=vs.85).aspx
Good luck! Please report back if it worked so other folks could enjoy it too 🙂
Ilantz
Hi Ilan Lanz (Ilantz),
I am running exchange server 2013 with DAG. I have two mailbox servers on which DAG is running. The problem is that my log files are not purging with Windows Server backup. I tried to enable Circular Logging and then dismount database and then mount it again. It did not have any impact on log files.
One thing which I noticed that during windows server backup the service for Microsoft Exchange Server Extentsion for Windows Server backup do not starts. If I manually start this service then it stops after few minutes.
Please help us as we are almost run out of the Disk Space.
Regards
Anees
To make sure circular logging will
take effect, make sure that you dismount and suspend all other copies. Mount back the db following with resuming the copies. The logs should truncate soon once all copies
are in sync.
Let me know this worked out.
Ilantz
This is a life saver. Just helped me fix filled disk space before my client noticed. Thks a lot for posting this
Sure! Glad it helped you!
Can I use this method in Exchange 2010 Service Pack 3?
Yeah sure.
Hi ILan,
Now I have a VM running Exchange 2010 SP3 in a Win2012 Failover Cluster. The BDs are store in 4 Dynamic VHDs.
Can I use this method in my VM?
Thanks in advanced.
Yes, this should work just fine. Make sure thus to backup the active server with the active databases.
It’s a VM in High availability not a guest cluster. My VM runs all the roles.
Great. Then yes it should work.. Post back if it did
It didn’t work. In my event viewer I found this error:
Source: VSS
Event ID: 8224
The VSS service is shutting down due to idle timeout.
This article still holds true today. Our full backup of Exchange 2010 via Backup Exec completed successfully but Exchange still showed that no backup had occurred for one specific database. The VSS backup didn’t work the first time I attempted it however. The VSS exchange replica writer service showed a retryable error. I activated the database on another DAG member, rebooted the server with the VSS error and then performed the steps above. This bought me plenty of time to resolve the issue with the full backup. Can’t thank you enough!
Thanks for your feedback Alex. I’m happy it helped you out!
Ilantz
Quick question Ilan Lanz…. one of my customers actually DELETED exchange LOG FILES (10 GB out of 30GB in the system)…. now store is still up but VSS backup fails due to consitency check beeing bad…. would you method be a good try? or you dont recomend it?
Well… If the sequence is no longer valid. You could enable circular logging just to “clean” everything up.
I’m not sure running the vssadmin will sort it out as it still calls the exchange vss writers…
Good luck!
I have tried this in my test environment. I have all virtual disk assigned to server, unfortunately It didn’t work.
What’s the topology like? Any particular event viewer entries?
Thanks for writing this article but unfortunately it did not work for me. There were no errors received, it appears to have run the shadow copy backup without a problem. Once complete however I don’t see any of the events listed in this article in the event log and the logs never clear. I am on Exchange 2010 SP2 with storage attached LUN’s.
I did tested that method on an exchange server with local disks, however I see no real reason why it won’t work with storage LUN’s attached…
If I’ll have the chance I’ll try to reproduce and report back.
Would appreciate if any one subscribed to the post will share their result with the storage config they’ve used.
Ilantz
This worked like a charm even with storage LUN’s attached!!!
Thanks a lot…great job.
P.S. i didn’t get same logs though. However, got event ID 2157 which says: The replication instance for database (name of database) has copied and replayed multiple logs.
After that checked space and boooom, around 40 Gigs of free space retrieved.
Thanks for the feedback HK ! glad to hear this method worked for you too 🙂
For those of you who claim that this does not work. I am here to save you. I too thought that this method did not work, however I was incorrect. The problem is that a lot of folks have their logs and database drives on different drives. Sound familiar? See below.
Where N is the database drive and M is the log drive.
Open an administrative command prompt from your exchange server and do:
diskshadow
add volume M:
add volume N:
begin backup
create
end backup
It was not working for you before because you need to include all the drives with exchange data on it (both the drives – data and logs in my case). Cheers!
Thanks for the feedback josh! I’ll add the reference in the post.
thus this require also duak space since it is a backup methid or this is just fir truncation no need for additional disk space.
thanks.
ian
This methods calls the VSS writer and mimics the “real” backup. The actual snapshot is not really created..
Just a note on enabling circular logging for a database with 2 or more copies in a DAG, you DO NOT have to dismount/remount the database for circular logging to take effect. Again, you must have multiple copies of a database for Continuous Replication Circular Logging (CRCL) to begin removing log files on the fly, and it may take up to 15 minutes for you to see the log files begin to disappear.
See Scott Schnoll’s excellent post on CRCL – http://blogs.technet.com/b/scottschnoll/archive/2011/06/27/circular-logging-and-mailbox-database-copies.aspx.
Thanks for the note! I’ll edit the post to make this more clear.
Is there any sort of script that could take the Exchange server offline, turn on circular logging, pause, then restart the Exchange server?? 10-15 minutes’ downtime is tolerable…
Why would you script that? Do you have multiple servers that need this on regular basis?
Worked perfectly on Exchange 2007
I love you 🙂
lol 🙂 happy it worked out for you !
ilantz
Worked perfectly on Exchange 2007 with local databases.
Thanks so much for this procedure!!!
Sure ! glad to hear it worked for you.
ilantz
Thank you, This saved me tons of time and energy.
Thank you so much for this quick fix!!
You’re welcome! Glad to hear this method keeps helping folks out 🙂
That worked great!. Excellent move. Thank so much for the post.
You’re welcome!
Wow wow wow!
I have been using the old fashioned way of dismounting, checking its health and moving out logs. This has taken the downtime at my clients from hours to a mere few minutes!
Thanks you!
You’re very welcome !
Thank you Very much ! This way worked on VM ESXI, disk in the storage(lun).
Thanks very very much!!
This greatly helped me get out of a critical situation.
Yomi Ogunbajo…….this works like magic and it has been tested on Exchange 2010 and exchange 2013 as well.
Note: Do make sure you add the other volumes that houses your Databases as long as its from the drive that houses the log files.
DISKSHADOW: add volume G:
DISKSHADOW: add volume H:
DISKSHADOW: add volume I:
DISKSHADOW: add volume J
DISKSHADOW: add volume K:
DISKSHADOW: add volume L:
DISKSHADOW: begin backup
DISKSHADOW: create
NB: Drive G is for the log file while Drives H-L are for the DB.
Yomi Ogunbajo…….this works like magic and it has been tested on Exchange 2010 and Exchange 2013 as well.
Note: Do make sure you add the other volumes that houses your Databases as long as its resides on a different drive from the drive that houses the log files.
DISKSHADOW: add volume G:
DISKSHADOW: add volume H:
DISKSHADOW: add volume I:
DISKSHADOW: add volume J
DISKSHADOW: add volume K:
DISKSHADOW: add volume L:
DISKSHADOW: begin backup
DISKSHADOW: create
NB: Drive G is for the log file while Drives H-L are for the DB. The create command takes a while to execute depending on the size of the log files and the DB in your Exchange Organization.
Great post!
Quick questions..
1 should work fine on Exchange 2013 right?
2 do these commands run quickly… ie no backup is actually run so the process is just a minute or two?
thanks in advance
Hi Chris,
1 – Yeah sure. Although I didn’t tested it yet, if you did – please report back !
2 – Should be super quick…
ilantz
Hi Ilan,
I actually did a test run on an E2k13 server and it worked perfectly.
Chris
Awesome ! thanks for reporting back
Hi Chris,
The fakebackup script works fine on Microsoft Exchange 2013 and it does not take time, when you initiate the create command that may take a while depending on the size of the logs and you DB!!!
Yomi Ogunbajo
This was exactly what I was looking for. Nice job!
Happy to hear this method worked out for you ! Thanks for the feedback.
Thanks for the info. I am right about at this stage since my backup server is loosing communication for some reason with the Exchange server. Now I have 40gigs left out of 1.5 tb. Exchange store is only 200gigs. My question to you is, will this method work if all stores are on the C drive? Can I do this spoof shadowcopy on the one and only drive (scary) that obviously also runs the OS.
Yeah. No worries. It’s merely just a “fancy” way to call the VSS writers to do their job.
Which is also the same way all of the “regular” backup solutions work.
Ilan
Thanks Ilan. One thing. It didnt complete. Thoughts?
DISKSHADOW> add volume c:
DISKSHADOW> begin backup
DISKSHADOW> create
The last operation failed.
– Returned HRESULT: 80042313
– Error text: VSS_E_FLUSH_WRITES_TIMEOUT
DISKSHADOW>
Worked Great! Thanks for the help.
Thank for the feedback ! Glad it worked out for you.
Ilantz
Your strategy worked great for me! Thanks for sharing it with us.
Thanks for the feedback. I’ve actually used it also myself this week on yet another case with about 1TB worth of logs that got purged 🙂
You sir are an IT legend! This is a thing of some beauty
LOL ! Glad it helped you out !
Absolutely GREAT! Thank you!!
You’re welcome!
Great stuff. Can you please make a note that the log files didn’t delete straight away, but about 30min later (for me at least).
Good point. Will do.
Wow, cool, thanks. This worked on 2016 as well for me. Great!
Thanks for the feedback !
Hi Ilan. Thanks for the really great and helpful article. Can you confirm one thing for me… When you run the DiskShadow cmd to create the spoof shadow copy does it actually create a backup of all the files on the volume that you specify and where is that backup actually stored? We have a 1TB drive where the Exc 2007 server logs are stored and its almost out of space and none of the other drives on the system have sufficient space to hold a backup of the logs. We don’t currently have Exchange specific backups configured (we’re working on that) and so the transaction logs are building-up. Your solution would be a good workaround until the backups are configured but I am bit concerned about what will happen when running diskshadow. I’ve googled the DiskShadow cmd but can’t seem to clarify if it actually backs-up the files to another location and, if so, where it puts those files? Does it just pretend to perform a backup or does it actually create a copy of the files somewhere? Hope you can clarify for me…
Hi Stuart,
Sorry for the delay. The command just pretends to perform a backup.
The original case that led me to write the blog post is just what you are facing 🙂
Ilantz
MB Shaikh,
Hi Ilan Great article, I have a situation one of the database logs folder is running out of space on Excahnge 2013 DAG member attached EMC LUN, and at the moment there is some issues with backup solutions, so can i use this method only for that particular database ? please advice. and as you mentioned Is it is must to do a backup ofter this procedure?
Thanks in advance.
Hi yeah, this should work. Keep in mind that this work in the volume level.
Because you are removing log files with this method. You must do a full backup because you lose the point in time capability when you purge log files.
Keep in mind that within a DAG the actual removal time of the log files is delayed. So wait a while after you perform the method.
Ilantz
Legit! Thanks a lot for this!
Thx for the feedback. I’m glad to hear this post helped you too 🙂
Thank you for this!! Worked on Exchange 2013.. had a server with over 1 TB in logs and this fixed it.
You’re welcome !
Great post, but I do have a question or two:
1) I see in some of the comments that this should work in a DAG environment, do you run this on each member of the DAG or just the active member?
2) in my environment I have several databases and logs stored on the same volume, will this process identify each database that is mounted by the Information Store and truncate log accordingly?
thanks again for a great post.
1- if all copies are healthy – doesn’t really matter where you run it.
2- this works on the volume level, so all should be “backed up”
It seems, that it didn’t work for ReFS Volumes.
I’ve mounted the volume on C:\ExchangeDatabases
If i run it on C: no logs will be truncated.
Thanks for raising this.
Didn’t work for me.
Exchange 2010 DAG server, I actually lost 100MB doing this..> AH!
Do note that for a database to be “eligible” for log truncation in a DAG – all copies need to be healthy.
Great job i clear almost 300 Gb without any problem (DAG)
Thanks for sharing ! should have added a counter to this post 🙂
I was desperate trying to purge the logs of my Exchange 2013 CU15, after moving hundreds of mailboxes to a new database, because I was having a problem with my backups, using the diskshadow, solved my issue and I was able to execute the backups successfully again.
Thanks Ilan!
So happy you found it useful. Thanks for the feedback
This worked for me finally!! Initially it did not work. I right clicked run as admin even after being admin and then it worked.
I just wanted to know what do you mean by You should preform a “Full Backup” right after this process is done. Are you referring to system image backup using acronis or any other third party software? And is it required?
My exchange 2010 databases are on mounts points. How should i mentioned drive letters in diskshadow? Drive letter where it is mounted or complete path of mount folder?
Mount points should work just like drive letters, just make sure you include both DB and LOG directories if you separated them.
Thanks! Works like a charm!
This didn’t work for us on Exchange Server 2016/2019. What did work is to use the PowerShell script seen on the post https://www.alitajran.com/truncate-exchange-logs-with-powershell/