Chaos

I have been a Perfect Disk user for some time now, recently I decided to try version 11 from Diskeeper. The one thing that I am not really sure of is why Perfect Disk does not make any changes to the size of the MFT yet Diskeeper 11 increases the size by large amounts with the built in Frag Shield utility. Can anyone explain why they do this and if it's a good thing or a bad thing

__________________

gshayes

PerfectDisk doesn't have this "feature" because it is not needed.

A little background on NTFS and how it works. When an NTFS drive is formatted, it creates the $MFT and it contains a fixed number of records that can be used. As files are created, then these records are used. Eventually, the pre-allocated number of records is filled up and the $MFT needs to "grow" - allocating another chunk of file records. Immediately adjacent to the $MFT is created what is called the MFT Reserved Zone. By default, it is 12.5% of the drive and goes from the first record of the $MFT to the first non-free cluster after the last MFT record. If you have a 100GB drive, then the Reserved Zone is going to be 12.5GB - pretty large if you think about it. If you look in Windows Explorer/Properties on a drive, the free space shown is the total of both INSIDE and OUTSIDE of the Reserved Zone.

The MFT Reserved Zone is created specifically to allow the $MFT to "grow" in a contiguous fashion. When the $MFT fills up, it allocates the next chunk from the free space located in the MFT Reserved Zone. NTFS will avoid putting files inside of the Reserved Zone unless you get into a low free space condition. It is not necessary to artifically pre-allocate additional space for the $MFT in order to keep it from growing fragmented.

Since NTFS takes care of this automatically, there is no need to artificially extend the $MFT. However, since most people don't have this level of knowledge of how NTFS works, when they hear about this "feature" in Diskeeper, they think it must be a good thing and worth paying extra money for.

- Greg/Raxco Software
Microsoft MVP - Windows File Systems

Disclaimer: I work for Raxco Software, the maker of PerfectDisk - a commercial defrag utility, as a systems engineer in the support department.

Warhammer

I always wondered about that I havent had much luck with diskeeper 10 it doesnt run very long running on a raid setup without eventually getting corrupt files causing and error in the raid array. Maybe just me but when I quit using it Ive had no more problems.

PangingJr

gshayes,
you might want to include the "quote" tag over the message that you post above.

mmaterie

Hi, I'm the product manager for Diskeeper. Let me clarify an earlier point and explain the purpose of FragShield better.

What Greg mentioned about the behavior of the MFT reserved zone is very correct for Windows 2000 (NTFS 3.0). In Windows 2000 a fragmented MFT meant you HAD to use a third party tool to run a special "bootime" defrag. ALL metadata files (i.e. the files that define the file system - of which the MFT is one) were unmovable objects.

However, in Windows XP/2003 (NTFS v3.1) Microsoft enhanced the APIs to support online defragmentation of most of the MFT (as well as several other metadata files). They also changed the design of the reserved zone. It is no longer a hard-coded region. In XP, that region is dynamic and is an NTFS internal hint. In XP that zone can also be defragmented into and out of - in Win2k you could only move files out of it.

A good freware tool you can use is to verify this is Windows Sysinternal's NTFSInfo.exe.
[COLOR=#800080]http://www.microsoft.com/technet/sysinternals/FileAndDisk/NtfsInfo.mspx[/COLOR][COLOR=navy][/COLOR]
The command line tool will output the percentage that the reserved zone is using.

What Greg mentioned about the MFT expansion is also correct.

FragShield's purpose mirrors what many database administrators (DBAs) do as a best practice. A DBA typically creates a large database in advance, even though the total number of records only consume a small percentage of the size of that file on disk.

Pre-allocating the MFT with FragShield offers similar capabilities of what a DBA would do for their database. If you have a Win2k system and know that you will need more than 12.5% of the volume, FragShield can pre-expand the MFT and prevent it from fragmenting - mitigating the need for a bootime defrag. Diskeeper will advise you on the need.

On an XP/2003/Vista system FragShield offers the same functionality. As the zone is more dynamic, fragmentation of the MFT often occurs more rapidly than in Win2k. If you install a large app (like a software suite or a game with many files), run a defrag analysis - you'll often see MFT fragmentation. Fortunately an online defrag can address this fragmentation, but using FragShield in advance will prevent the fragmentation from occurring in the first place.

Is this feature "life or death" - of course not. Does this feature provide value - I think so, and so do many others. If it is of value for you, that is up to you to decide.

BTW, FragShield is available in the Diskeeper Professional product and higher editions. The only edition that does not have this feature is Home.

mmaterie

Warhammer,
Diskeeper, like other defragmenters, uses the Window's APIs to perform online defrag. If you're getting corruption issues, you may have a faulty controller. You're always welcome to contact Diskeeper technical support.

Also note, that if you run Win2k there is a known issue with the file system generating false positives (it is very rare though):
http://support.microsoft.com/default...;en-us;q320866

Diskeeper and the built in defragmenter regularly check the file system health (other programs may have similar functionality), so a false positive can unfortunately complicate defragmentation in some cases.

temeteus82

I tested the trial of diskeeper 10, some time ago, and found that it can defragment my iPod mini (4Gb) does mp3 player that uses hdd to bee defragmented?

__________________
How you can help HardwareHeaven by using Digg!

Chaos

thanks for the input, I am surprised to see Raxco and Diskeeper employees replying but if it comes from the source I guess I should listen.

Warhammer I used version 10 and had issues with corruption of files also.

MMaterie. is there a way once Frag Shield has increased the default size of the MTF to restore it to it's original size after version 11 has been removed? Also do i read correctly that the increase of the MFT size is precautionary only and will provide no real benifit unless the default limits of the MFT are surpassed.

__________________

Asmoday

I would like to say on behalf of the driverheaven staff that it is very nice seeing Raxco and Diskeeper reps here. It is very positive with companys that actually care about the end users.

Welcome to Driverheaven gshayes and mmaterie!

__________________

Jeremy of Many

Could the employees from both PD and DK please explain which is technically better/healthier for the harddrive...
a) Having files placed side-by-side in sequential order beginning at the beginning of the disk spanning outward (SmartPlacement)
b) Having DK automatically defragment everything for you, and using I-FAAST?

I mean, both defragmenters claim to be the leading best in the industry, yet they both operate in two completely different manners. One of them has to be better than the other, right?
SmartPlacement's main con is the extra time and workload stressed on the harddrive to constantly move files around in order to make room for say, newly installed programs.
And DK says the Invisitasking is real-time, that it eliminates fragmentation AS the file is being written to the disk. This isn't true because I have copied archives and immediately done an analysis and it would be in many fragments which wasn't defragmented until later.
These concerns need honest, unbiased explanations. People need to be told the absolute truth.

Cheers,
Jeremy

BWX

I haven't been able to get a boot time defrag to work since PD6.. I have heard many other people with the exact same issue. It doesn't work on the main C drive or the drive (partition) you use for the pagefile. I use a separate partition.

it works on drives/ partitions that can me taken off line to defrag within Windows. - basically it doesn't work the drives / partitions you need it to work for.


Last pic I took of the error. PD8 doesn't work either.


Using a regular IDE drive on Asus A8N-E.. and Nforce 4 Ultra mobo.. doesn't work no matter if I use default windows driver for IDE controller or the Nvidia drivers. I use the windows drivers because they are less buggy for me in general.

I had an older HD die on my a while ago (about 4 years old- WD 40GB 7200PRM 2MB cache ATA100)... I can't help but wonder if I defragged it too much.. it just started clicking one day at startup and my PC wouldn't recognize it. Or maybe it was just "it's time" to go.

Warhammer

I run it on my wife and daughters machine without issue but using it on a machine with a raid 0 array after a month or so you end up with corrupted system files. This happened on my Asus A8v deluxe and now my Asus A8n sli deluxe. Maybe it doesnt like Asus boards I dont think the controllers in both boards would be bad. I dont know about other arrays but I havent had any issues since not using it but I would really like to learn more because Ithink its a good program.

mmaterie

No. This is a one way road - shy of copying the data off and reformatting. Unfortunately the MFT is relatively off limits to third parties, otherwise quite a bit more functionality would be available. Databases and virtual disk formats can typically be shrunk with a vendor provided routine, for good reason. I can't think of any technically substantial reason for shrinking the MFT though - at least not off the top of my head.

Increasing the size is simply to prevent fragementation. That is possible because for this unique file, NTFS actually does leverage a reserved area for growth. From a performance standpoint, a handful of MFT fragments is not an issue. A good part of the MFT is cached. Same goes for most all other metadata on a volume as most of those files are always in cache after the volume is mounted.

The primary benefit in defragmenting metadata is to prevent other data files from fragmenting around them.

mmaterie

Defragmenting an HDD-based MP3 player won't hurt (I've defragged mine several times), though a good deal of the "impact" of fragmentation is relative to how the operating system is affected by it. I haven't seen a scientific test that defragmenting an iPOD helps, but I have seen a lot of anecdotal evidence. I wouldn't, however, regularly defrag a SSD (solid state disk) such as the iPOD Nano though, as the "write" life of those devices is far more limited.

mmaterie

Hi Jeremy,

If you want more info on I-FAAST, I've written a number of articles at www.diskeeperblog.com. I also wrote a blog on the real-time defrag - you have a very valid question.

I also agree with your demand for forthright info, and I think both vendors here are up front with technical details. Yes, marketing product webpages tend to be "value" focused, so you may have to dig into the vendor's white papers or FAQ sections to get to the specifics.

You are correct that the two strategies are different, and they have different objectives.

I'll let Raxco, the experts on their product, define how their technology works.

-Michael

mmaterie

I'm not the Raxco expert, but that error indicates some service likely started early in the boot process and is preventing the bootime engine from exclusively accessing the volume. Diskeeper behaves similarly and would run into the same limitation. If you can track the service down and edit it's start up value, it should resolve the issue.

Here is a good reference as to the general cause:

http://support.microsoft.com/default...;en-us;Q235771

I'd certainly recommend using your service manager to edit, rather than editing the registry - if it can be avoided.

-Michael

mmaterie

Hi Warhammer,

I have the exact same board as you (and the Intel ASUS equivalent on another box). I've also set up a 2 drive RAID 0 with the one of the onboard SATA configs (I think there were 2 separate on board RAID controllers available?), but have never seen an issue. I am, however, running x64 XP and using Windows supplied drivers updated from the Windows Update website. I suggest emailing Diskeeper tech support - they can investigate.

-Michael

PangingJr

Could you provide example of services that starts ealier than the boottime defrag at system startup?

if i can recall correctly, one of ZoneAlarm's previous version was also have this similar problem with either Chkdsk or boottime defrag (i can't remember if it was about the boottime defrag or not). also the Symantec AntiVirus Corp 9.x used to have peoblem where it can cause the Chkdsk not to run at system startup and at the time i found this issue and i also found that Symantec has came up with a workaround to this by recommending users to configure Auto-Protect to load at service startup, instead of at system startup.

BWX

I would like to know what kind of services would start up too soon so I could temporarily shut them down so I could run a boot time defrag.

I use AVG free, and Sygate pewrsonal firewall pro..

mmaterie

From personal experience (I used to work in the security industry for a major AV vendor before joining Diskeeper) most issues with premature service startups are from file filter security products, and are unecesarry. Firewall, Encryption and AV tools are your best suspects (anything "device driver" oriented). Please note I use "premature" loosely as it is always possible there is a legitimate reason for early start up. If you plan to explore and fix by yourself, you can likely narrow your search to third party software registered there, or do a search for the value 1 in this section. I'd double check with the vendor of the related service that editing the startup is OK to do.

PangingJr

i just thought you may have some of the third party service names in the KB issues in the Diskeeper software,
however, thanks for explaining that to this thread. i know you have helped answering this problem for other software product.

Jeremy of Many

mmaterie, will you read my post here:
http://www.msfn.org/board/index.php?...c=85812&st=15#
and provide a response to it? I'd greatly appreciate it.

__________________
[b]Antec P180 | OCZ PowerStream 520W | DFI LanParty NF4 Ultra-D | AMD X2 3800+ @ 2.4 GHz | 160GBx2 + 500GB Seagate SATA2 NCQ 8MB & 32MB | 2GB OCZ PC3200 DC (2.5-3-2-5) | GeForce 6800GS 256MB | Sound Blaster Live! 5.1 | Razer Copperhead Laser Mouse | Standard M$ 101/102 Keyboard | Windows XP Pro SP2

PangingJr

Jeremy of Many,
read your post there, as you said "One of them has to be better."...in someone's mind.
you seem to like the Diskeeper more than PerfectDisk, already. no?
however, perhaps you could repost your questions here? so if you ever got any response from either of the experts...
then we can read both questions and answers altogeter a bit easiler, more understandable and we all get the benefits of that.

Jeremy of Many

The simplicity of my understanding so far is:
PerfectDisk will take more time and put a heavier workload on the harddrive by using the SmartPlacement method; placing all files together in sequential order from the beginning of the disk spanning outwards. One pass does usually leave 0 file fragments remaining, so it is deemed very efficient. However, one reboot later and you have fragmented files again, anyway! Here's a scenario: Let's say for example if svchost.exe was placing near the beginning of the disk, for quickest access. The user then goes to Windows Updates and that file is overwritten with a newer patched one. Does PD move all files out of the way and squeeze that file where the previously written one does, putting an extraordinary amount of work on the drive for one file, or does it just place it at the end of the line, defeating the purpose of the previous task of placing it near the beginning for quickest access? This is something I should actually ask here, since employees from DK and PD are replying there. It would be wonderful to get clarification as to what happens in this scenario.
Diskeeper, while it doesn't do SmartPlacement, there is I-FAAST 2, which you can learn a lot about by reading Michael's (Project Manager of Diskeeper) blog entry. It does invisible (InvisiTasking) and completely Automated defragmentation in "Real-time", which you can understand thoroughly by reading another blog entry of Michael's. I previously used PD in the past when I truly believed it was superior in its efficiency. However, since DK2007 was released, I cannot be bothered to manually defrag my files. I would rather have DK manage my files in the background at no expense to my overall system performance than to have PD defrag them nicely during one session, let them fragment in 24 hours or two weeks (as users are frequently doing various tasks which result in varying levels of I/O (Read/Writes) then defragment them again.
In the long run, I think Diskeeper is more beneficial to the lifespan/health of the harddrive, the reason well-explained in the second blog entry of Michael's I provided a link for above, or again here.

Don't get me wrong, I'm not trying to be a fanboy here. If the experts at Raxco can prove in detail that PerfectDisk is better and healthier for a harddrive, and explain both the pros and cons of using PD8, than that would be the best thing for everyone here.

__________________
[b]Antec P180 | OCZ PowerStream 520W | DFI LanParty NF4 Ultra-D | AMD X2 3800+ @ 2.4 GHz | 160GBx2 + 500GB Seagate SATA2 NCQ 8MB & 32MB | 2GB OCZ PC3200 DC (2.5-3-2-5) | GeForce 6800GS 256MB | Sound Blaster Live! 5.1 | Razer Copperhead Laser Mouse | Standard M$ 101/102 Keyboard | Windows XP Pro SP2

gshayes

There are several known 3rd party programs that have drivers that will prevent PerfectDisk's boot time defrag from running. The Raxco support site at http://www.raxco.com/support has detailed information about the programs that we are aware of and if the 3rd party developer has released. The PerfectDisk Support Info (Help/About PerfectDisk and then click on the Support Info button) looks for the ones that we are aware of and tells you if they are found.

- Greg/Raxco Software
Microsoft MVP - Windows File Systems

Disclaimer: I work for Raxco Software, the maker of PerfectDisk - a commercial defrag utility, as a systems engineer in the support department.

PangingJr

Sorry that i asked you to quote your own messages before, i didn't know that was you who posted that.

and thanks for the information on the Support Info button, this is new to me too.

here is the name of third party programs...

gshayes

Quick question for you. What is the difference between scheduling a daily defrag pass - which will automatically defragment your drive everyday - or having something run continously every day? The answer? Not much - except that you have more control over defrag activity when you decide when/if it runs.

Probably the biggest difference between Raxco's defragmentation strategy and the way that other defragmenters work is that PerfectDisk is designed to do the best job possible in a single pass - at the end of it's pass, the job is finished/done - you don't have to wonder if the drive is better or wait for who knows how many passes (or how long if running all of the time). PD is also designed for worse case drives - where available free space is low and fragmentation complexity is high. For years, PD has been the only single pass defragmenter out there. PD was designed from the beginning (based on our proven technology originally developed for performing disk optimization for OpenVMS systems) to improve drive performance in a single pass - meaning defragmentation of files and consolidation of free space - while being "friendly" with usage of system resources. PerfectDisk was the 1st defragmenter to effectively and effeciently defragment multi-TB drives (which even the workstation version of PerfectDisk can easily do - nothing special needed). We still remain the only defragmenter that not only tells you about all of the NTFS metadata but also defragments all of these "special" files. We remain the only defragmenter that defragments the hibernate file. We actually provide information on free space fragmentation (most defragmenters don't even though they talk about the importance of free space consolidation - even if they don't effectively do it) .

Keep in mind that the ONLY reason that you defragment a drive is to improve drive performance. The best drive performance possible can only be achieved by improving read performance (it's faster to read a defragmented file than a fragmented file) AND improving write performannce by consolidating free space on the drive. PD focuses on BOTH of these aspects of drive performance - not just improving read performance (I don't know of too many Windows systems where most of the disk activity is simply read activity). Kinda of like putting low-test gas in a high performance race car. Sure, the car may run - but it doesn't "zoom" like you'd expect it to

Raxco is a close Microsoft partner. Besides me being a MS MVP and having direct access to the file system group, our developers also have a very close relationship with the MS file system team - they are all PerfectDisk fans. Every version of PerfectDisk beginning with V4.0 has been certified by Microsoft and Raxco was on the team that helped Microsoft write the Vista certification requirements. Microsoft itself talks about the importance of consolidating free space as part of defragmenting. How important? The free space consolidation portion of the Vista defragmenter was completely re-written to do a more effective job of free space consolidation. A real good "read" is a test that was performed several years ago by one of the original developers of NTFS on the performance improvment that can be measured and seen by consolidation of free space (vs not consolidating free space) - which can be read at http://www.raxco.com/products/perfec...solidation.pdf

As far as a file placement strategy, Smart Placement (patented) is primarily designed to accomplish 2 things: (1) Slow down the rate of re-fragmentation (remember, the file system is designed to fragment. You can't prevent it from happening but you can slow it down) and (2) Speed up future defrag passes. Heavier workload using Smart Placement? Possibly - but not always. Remember, Smart Placement is doing several things - defragmenting files, intelligently placing them AND consolidating free space. Compared to defragmenters that are doing just defrag or possibly defrag and some sort of file placement Smart Placement may take longer but ultimately, the drive performs better in the long run.

Smart PlacSmart Placement is based on file modification date - NOT last access date. With Smart Placement, once an initial defrag pass has been performaned, future passes require less effort (both to defragment files and consolidate free space). There are some additional performance benefits that can be seen (improved boot speeds) due to where Smart Placement places boot files (at the "beginning" of the logical drive adjacent to the boot sector). However, Smart Placement doesn't attempt to try to get further improvement in drive performance by specifically placing files at a particular place on the drive where it is presumed that the drive is fastest (presumed but very difficult to accurately determine as the drive typically isn't idle for a long enough period of time to accurately detect - works great in the lab but not so well in a production environment). Typically, the "cost" to determine these files and shuffle them back and forth on an active drive where what is being accessed flucuates over time as the drive is used isn't worth the very minor performance improvement that might be gained. You get a far better improvement in drive performance from free space consolidation than specific file placement.

Regarding PD8 specifically, couple of new things. Single file defrag (allows you to quickly defragment a single file) as well as a Consolidate Free Space defrag (defragments files and consolidates freespace - but doesn't Smart Place files). Our patent-pending Resource Saver technology also reduces the system resources required to defragment a file by eliminating some of the initial I/O usually required prior to actually defragmenting a file. As drives get larger and larger, being able to quickly and effectiently defragment huge drives is going to be very important. PD8 is designed for these types of drives.

What's nice for defrag users is that people actually have a choice - there are several vendors - from home user "shareware" defragmenters to full featured enterprise enabled defragmenters like PerfectDisk and Diskeeper.

Well, I don't know how many characters I'm allowed to post so I think I'd better quit now and see if it is accepted

- Greg/Raxco Software
Microsoft MVP - Windows File Systems

Disclaimer: I work for Raxco Software, the maker of PerfectDisk - a commercial defrag utility, as a systems engineer in the support department.

mike2h

just wanted to thank everybody involved for a very informative(& professional) thread.
have been using pd since 6 on all 3 comps. very happy with it but have been considering trying dp just becuase i like to chx my options. always wondered what the main difference was, this thread has taken care of that.

__________________

BWX

Thanks I found the ones that I have installed. I will try the work-arounds..

http://www.raxco.com/support/windows...d=537&issue=41

Jeremy of Many

Is that a represented-as-a-whole opinion of PerfectDisk, or just yours?
A daily defrag pass, or automatic invisible defragmentation? Sure, a user can schedule a defrag at, for example, 5 PM because they intend on being AFK. Let's say their plan changes and they end up sitting down at the PC at 5:03 PM. They open up their web browser or video editing software, but then remember that PD8 is defragging their drive. It has already analyzed the drive, so what happens to a file they modify, delete, or create while the defrag pass is in session? Is it dynamically dealt with, does PD ignore it, what?
My point is, they may be in control of when defrags take place, but I'm asking what happens to the files that are changed during a defrag pass?

I realize from experience that PerfectDisk is very efficient in one pass. I've thought this through for a good portion of the day and I understand how, when combining PD's method of SmartPlacement and Free Space Consolidation are designed to prevent future fragmentation. However, one drawback (con) I can think of is, given the scenario I previously mentioned:
Let's say for example if svchost.exe was placing near the beginning of the disk, for quickest access. The user then goes to Windows Updates and that file is overwritten with a newer patched one. Does PD move all files out of the way and squeeze that file where the previously written one does, putting an extraordinary amount of work on the drive for one file, or does it just place it at the end of the line, defeating the purpose of the previous task of placing it near the beginning for quickest access?
Would you be so kind as to clear that up for us?

Before quoting the following, keep in mind that I did not edit your words in anyway, I simply restructured it in list form to make it more easily readable:
If Michael has anything to say about Diskeeper's lack of or similar ability to do some of that, that would be nice.

As I stated in the beginning, I realize the importance of this. However, both PerfectDisk and Diskeeper do this. I've read the PDFs, but I question their legitimacy, especially when they are done by Raxco and not a 3rd party. Also, the one of FSC is outdated and doesn't address newer versions of either programs.
The Impact of Free Space Consolidation On Windows File System Performance PDF and Michael's post really seem to oppose each other. Raxco says Diskeeper results in wasted seeks (I/Os), and although Michael's post isn't directed at Raxco, it does provide a good understanding and emphasize the importance of defragmenting a harddrive in order to expand the lifespan on a harddrive. So those two sources do, in a way, contradict one another.

Hey, I prefer efficiency over speed any day. Thus far, we've established that Free Space Consolidation is essential, therefor without it, defragmenting alone doesn't fully amount to file performance. Michael, does DK2007 consolidate free space as well as automatically defragment?

Roger that.
No need to repeat yourself.
Yes, I have experienced better boot-times with PD in the past.
Isn't that exactly what SmartPlacement does? If going by frequency of usage and a file is recognised as not being used as often as it used to be, does PD not "shuffle" change it's color in the Disk Analysis and move it elsewhere on the next defrag? I've experienced this many times. You've really got me confused now, have my eyes deceived me? .....
Isn't this difficult task done by Diskeeper's I-FAAST feature?

Yes, I figured the term spoke for itself, but thanks for that brief dumbed down explanation.
If FSC is maximizing the amount of free space between the last file and the end of the disk, and SP already incorporates FSC, then why did Raxco make FSC Defrag a seperate feature?
I think DK beat you to the punch on that one.
Isn't it designed to take it's time for that extra efficiency that gives it that "single pass" edge? :P

Yes, and I'm trying to determine, hopefully with the help of honest people like Michael and yourself, which defragmentation software is the very best to use.

One more thing, I believe that your original post contained a statement which read something along the lines of "remember, the file system is designed to fragment", which you edited out. Care to explain that one?

__________________
[b]Antec P180 | OCZ PowerStream 520W | DFI LanParty NF4 Ultra-D | AMD X2 3800+ @ 2.4 GHz | 160GBx2 + 500GB Seagate SATA2 NCQ 8MB & 32MB | 2GB OCZ PC3200 DC (2.5-3-2-5) | GeForce 6800GS 256MB | Sound Blaster Live! 5.1 | Razer Copperhead Laser Mouse | Standard M$ 101/102 Keyboard | Windows XP Pro SP2