🔗 Source text from:https://akiba-pc.watch.impress.co.jp/docs/topic/special/2005307.html
With the continuous improvement of PC storage device performance, SSD speeds in the PCIe 5.0 era have approached 15GB/s. However, many people may not be aware that this high-speed performance is almost impossible to fully utilize in Windows Explorer. Even if the built-in SSD speed is fast, when copying files in the resource manager, the speed can often only reach a few GB/s, making it difficult to break through the bottleneck.
Although performance that meets specifications can be seen in benchmark testing, that is only the result in the testing environment. When using the resource manager for file operations in practice, the speed is far below the theoretical value, resulting in the underutilization of the performance of high-speed storage devices and becoming a kind of "waste".
This situation not only exists in existing PCIe 4.0 and PCIe 5.0 devices, but also persists even when using RAID cards to connect multiple SSDs to increase speed, or connecting external SSDs through Thunderbolt 5. Even if higher speed specifications emerge in the future, as long as Windows Explorer remains a bottleneck, this problem cannot be solved. With the improvement of storage device performance, the gap between user expectations and actual experience will become increasingly large, and this impact will become more apparent.
Mr. Hiroaki Shiramizu, Director and CEO of FastCopy Research Institute LLC, was interviewed at ITG Marketing
Background of FastCopy's Birth
It is precisely this situation that has made the developer of "FastCopy", Mr. Bai ShuiQizhang, dissatisfied. He developed tools that can fully utilize the performance of storage devices through unique design concepts, while achieving high reliability file replication and backup functions.
In November 2024, at the "Inter BEE 2024" exhibition, Samsung showcased a machine capable of achieving a sequential read speed of approximately 59GB/s at its booth, which was astonishing. At that time, this machine was mainly used to demonstrate the smoothness of 8K video editing, but Mr. Shirakawa pointed out that even though the high-speed performance of storage devices makes the editing process smoother, if the transmission (copying process) of material data and final data cannot be synchronized and improved, the overall efficiency still cannot be truly improved. Because the speed of data transmission in the resource manager is much lower than the theoretical performance of storage devices.
Lenovo ThinkStation P8 customized model showcased at Inter BEE 2024
At this point, the recorded continuous reading speed reached an astonishing nearly 59GB/s.
FastCopy played an important role in this scenario. In the demonstration at the exhibition, FastCopy was used for high-speed file transfer, and excellent performance was demonstrated through data transfer between the HighPoint external hard drive box "RocketStor 6542AW" equipped with Samsung data center grade SSD and the Lenovo ThinkStation P8 mentioned above.
CDFP connection external cabinet "RocketStor 6542AW" exhibited at Inter BEE 2024
The benchmark sequential reading speed is approximately 27GB/s
The speed of copying files using FastCopy is approximately 14.5GB/s, which is quite fast.
FastCopy is a tool used in Windows systems to copy (or delete) specified files or folders to a target location. Personal use is completely free, while the "Pro version" with commercial features requires a paid purchase. FastCopy has not only been widely used in the field of video production, but also adopted by many large enterprises and government agencies at home and abroad. It is a globally recognized software. Mr. Baishui also developed the LAN messaging software "IP Messenger" launched during the Windows 95 era, which many users may have used before.
FastCopy not only has simple copying functions, but its "Pro version" also compares data by generating hash values to verify whether files are copied correctly, thereby improving the reliability of data transmission.
FastCopy has existed since the era of Windows XP. At that time, Mr. Baishui found that the resource manager could not fully utilize the performance of the hard drive, and there were many inconveniences in backup and other operations. It was precisely out of dissatisfaction with these issues that he began the development of FastCopy with the motivation of "fully utilizing performance and achieving stress free backup".
When I tried to copy files on a 7GB/s PCIe 4.0 SSD using Windows Explorer, the speed was only around 2.5GB/s.
This is the speed when copying data in Windows Explorer using two SSDs with read and write speeds exceeding 5GB/s, but the speed is only around 2.5GB/s.
Why can FastCopy achieve high-speed file copying?
Mr. Baishui pointed out that there may be multiple reasons for the slow speed of Windows Explorer in handling file copying. Although he cannot fully determine its internal processing mechanism, he observed that read and write operations seem to be unable to be efficiently executed in parallel during file replication. For example, in resource managers during the Windows XP era, one operation was completed before another operation began, and even during large-scale data replication, operating system file caching became an additional overhead. Although parallel processing mechanism has been introduced since Windows Vista, current resource managers still cannot fully leverage their speed advantage, mainly due to the lack of direct I/O (bypassing the operating system cache mechanism) and insufficient initiation of asynchronous I/O operations.
It seems that asynchronous I/O operations are rarely performed in Explorer.
Based on these observations, Mr. Baishui proposed the design concept of "setting dedicated threads for read and write operations separately and increasing buffer capacity", thus developing FastCopy. After about half a year of development, FastCopy was officially released in 2004 and continuously optimized in subsequent versions, adding features such as asynchronous I/O processing and direct I/O operations.
In addition, FastCopy also considers the seek time of the hard drive and designs optimization algorithms to ensure that data is written to the location with the shortest seek time. It also adopts different replication strategies based on different situations (such as between different drives, between the same drive, or between HDD and SSD) to achieve optimal performance. For NVMe SSDs, due to their high upper limit on the number of queues (initiating processing tasks), FastCopy has also been optimized to fully utilize their performance. It is worth noting that although a large number of queues may cause issues with HDDs, the Windows system allows applications to identify whether the storage device is an SSD or HDD, so FastCopy can adjust its operational strategy based on the type of target storage device. In contrast, resource managers seem to have not fully utilized these technologies in this regard.
Collaborate with Samsung to achieve further high-speed development
The latest version of FastCopy is v5.8.1, released in January 2025, but Mr. Baishui is currently developing an undisclosed v6.0.0 version. The development of this version began with Mr. Baishui mentioning FastCopy's demonstration at the "Inter BEE 2024" exhibition on social media and stating that "there is still room for further speed improvement". Subsequently, Samsung, agent ITG Marketing, and ASK Company took the initiative to provide testing equipment and technical support.
Customized Lenovo ThinkStation P8 as a testing device
The testing equipment includes a customized Lenovo ThinkStation P8 computer. In the demonstration at the exhibition, when using FastCopy to copy approximately 20GB files from an external hard drive enclosure, the average transfer speed was about 14.5GB/s. Although it was much faster than the resource manager, it was still lower than the benchmark software's displayed read speed of about 27GB/s and write speed of about 24GB/s.
In order to identify the reasons why the storage device performance was not fully utilized, Mr. Baishui conducted in-depth testing using this "Lenovo ThinkStation P8". This computer is equipped with AMD Threadripper Pro 7975WX processor and 8 Samsung 990 PRO SSDs, connected through HighPoint's RAID card "R7608A", making it a top of the line configuration.
HighPoint RAID card "R7608A"
Equipped with large heat sinks to cool SSD and RAID controllers
The installed SSD is 990 PRO 2TB x 8
The test results show that in the ultra high speed I/O environment of tens of GB/s, the synchronous processing process within the operating system becomes the bottleneck, rather than the asynchronous read and write operations of data itself. A CPU core is fully occupied, preventing further asynchronous I/O operations from being initiated. This means that in a HighPoint RAID card environment, the transfer speed of a single CPU core can only reach about 28GB/s, while in a regular Windows RAID 0 (striped volume) environment, the speed is even lower, only about 13GB/s.
To address this issue, Mr. Baishui introduced a "thread pool specifically designed for I/O processing" and allowed multiple asynchronous commands for reading and writing to be initiated simultaneously from this thread pool. In addition, he also implemented a mechanism that "regardless of the order in which commands are initiated, as long as a thread completes I/O processing, the next I/O operation will immediately begin".
In the end, the core part of FastCopy was almost completely rewritten, including modifications to the underlying libraries. Although the changes were significant, Mr. Baishui had roughly identified the speed bottleneck and completed this version within approximately 35 days of development.
On the testing equipment, two sets of RAID cards were ultimately used for testing to ensure that both read and write speeds could reach the highest level. As long as there is no thermal throttling, the file copying speed can approach the benchmark software's recorded read speed of 59GB/s and write speed of 53GB/s, almost reaching the device's maximum performance.
FastCopy upgrade allows copying files at speeds exceeding 50GB/s. The number (109977) in parentheses after 52154 in the TransRate section represents the total read and write speed.
Confirm from Windows Task Manager that the data transfer speed exceeds 55GB/s.
The improvement of FastCopy enables file copying speeds to exceed 50GB/s. In the task manager, it can also be seen that the data transfer speed exceeds 55GB/s.
Mr. Baishui said, 'The device I used before could only reach a speed of around 20GB/s, so I didn't realize that there would be such a problem with ultra high speed I/O processing.'. But with the continuous improvement of user device speed, this situation will happen sooner or later. This attempt is very meaningful in order to tap into the potential of the equipment
Next speed bottleneck: 90-100GB/s
Although the release of FastCopy v6.0.0 will take some time, new speed bottlenecks have emerged. Even with the technology in the current testing version, the upper limit of transmission rate is about to be reached.
The next speed obstacle is expected to be around 100GB/s, and CPU and memory performance will also have a significant impact.
One of the reasons is the limitation of memory bandwidth. During the process of copying file data, the memory needs to perform both read and write operations simultaneously. For example, if you want the transfer speed of the storage device to reach 70-85GB/s, then the memory bandwidth needs to reach twice that, which is 140-170GB/s. Although such memory bandwidth can theoretically be achieved through Ryzen Threadripper or Xeon platforms, it is difficult to achieve this value in practical measurements.
In addition, even if the memory bandwidth issue is resolved through hardware upgrades, another problem is that the threads responsible for queue initiation are difficult to parallelize. At around 50GB/s, the occupancy rate of one CPU core has reached 50-60%. If it reaches 90-100GB/s, the CPU core occupancy rate will reach 100%, and the speed cannot be further improved.
Mr. Baishui stated that he is considering breaking through the upper limit of the algorithm through methods such as distributed processing. However, another issue is gradually becoming apparent: when the verification function is enabled during the replication process, the processing time will significantly increase.
The "xxHash" hash algorithm used by FastCopy is already very fast, but when processing large amounts of data, the speed limit is about 14GB/s. Therefore, he has started to work on changing the hash calculation part to parallel processing.
Adding a verification process can improve reliability, but the current bottleneck is the time required to calculate hash values.
We have confirmed that parallel computing through hash trees can achieve speeds of approximately 130GB/s. However, if data processing and hash calculation are performed on both the replication source and target simultaneously, with each processing speed reaching 50GB/s, a total of 200GB/s of memory bandwidth is required. The hardware threshold to meet this requirement will be very high. ”Mr. Baishui said.
The current DDR5-6400 memory bandwidth is only slightly higher than 50GB/s (for single channel), and even the test machine ThinkStation P8 equipped with 8 DDR5-4800 memory slots only measures a memory bandwidth of 190GB/s. From this, it can be seen that even simple file read and write operations can be a very "daunting" task for ordinary consumer hardware if one wants to "go all out".
FastCopy's advantages on regular storage devices
Although we are discussing high-speed transmission of tens or even hundreds of GB/s, in reality, the replication speed of resource managers has become a bottleneck even for hardware that ordinary users can access.
A high-end desktop computer built using ASUS ProArt components prepared on the day of the interview (CPU: Core Ultra 7 265K/motherboard: ASUS ProArt Z890-Creator WIFI).
The built-in SSD of the PC is the Samsung 9100 PRO.
We have prepared a Thunderbolt 5 casing and measured the speed
The shell is equipped with 990 PRO
On the day of the interview, a high-end desktop computer (CPU: Core Ultra 7 265K/motherboard: ASUS ProArt Z890-Creator WIFI) assembled using ASUS ProArt components was tested. The built-in SSD of the computer is the Samsung 9100 PRO, and the external SSD connected via Thunderbolt 5 is the 990 PRO.
In the test, the theoretical file transfer speed between these two storage devices should be 80Gbps (about 10GB/s), but when using a resource manager for transfer, the actual speed cannot even reach 2GB/s.
However, when using FastCopy for transfer, the speed can exceed 6GB/s and the processing time is only one-third of that of the resource manager. This efficiency difference is obvious. The advantages of FastCopy are particularly evident when dealing with large data transfers or regular backups, and it is worth actively using.
The benchmark test exceeded 6GB/s.
When using the resource manager to copy data, the measured speed is less than 2GB/s.
Using FastCopy, I am able to replicate at speeds exceeding 6GB/s, approaching the benchmark speed.
I confirmed from Windows Task Manager that the speed exceeds 6GB/s.
In the task manager, it can also be seen that the data transfer speed exceeds 6GB/s when using FastCopy.
Since the era of Windows XP, although the replication function of the resource manager has been improved, the actual test results show that the optimization for high-speed devices is still far from enough. Mr. Baishui expressed his willingness to provide FastCopy technical support to Microsoft if it could help improve. The future development of FastCopy is worth looking forward to.
Experience the high-speed performance of FastCopy
On April 12th (Saturday) and April 13th (Sunday), 2025, at the "ASK ★ FES 2025" event of Akihabara UDX, the ITG marketing booth will showcase FastCopy's high-speed file transfer demo machine. Visitors can experience its excellent performance on site and are welcome to visit.
Published on 2025-04-29
广东
FastCopy Pro Windows File Copy Backup Delete Software
