C
Camille Kokozaki
Guest
By parallel world, we are referring to the parallel computing and the file system world and not the one about the existence of which some physicists are speculating. Now that we have the title out of the way, let us review emerging new capabilities that are helping improve the throughput and performance of traditional EDA-centric workload deployment and execution.
A typical EDA workflow consists of users submitting jobs through job schedulers that distribute tasks for execution to compute nodes that interact with (increasingly clustered) storage systems in order to complete the intended tasks in the most economical and expedient way.
View attachment 9012
The design steps can be typically grouped into four main phases:
1. Architect from a concept or specification a high-level structure that identifies functional blocks and how they interact.
2. Design and verify the functionality and timing performance of the intended features.
3. Implement and realize the design in its physical structure and connectivity and re-verify that the timing can still be met.
4. Manufacture the completed design that has been transferred (taped out).
In each phase of the design steps, the workload profile varies in the amount of random or sequential reads, writes, or mix and intensity of data versus metadata (that is, data about the data). The analysis (characterization) of this workload can provide interesting insight into how storage and file system versions (such as NFSv3, NFSv4.1/pNFS) affect the performance of the jobs.
Network file systems (such as NFSv3, common in EDA environments) traditionally have the metadata and data share the same I/O path. In a pNFS protocol (a minor revision of NFS4.1) the metadata is separate from the data path. Every node in the clustered Data ONTAP® environment is a metadata and a data node for a pNFS client. In clustered storage architectures such as NetApp® clustered Data ONTAP 8.2 with pNFS support, data paths are local to all nodes. Data volumes can be moved non-disruptively, and metadata traffic can be load balanced.
This is how data access occurs: pNFS clients are directed to the physical node that hosts a specific file. Data access over pNFS is therefore always direct. If a volume is moved from one physical node to another, the pNFS clients will send I/O requests to the new location.
Taking the example of a simulation workload, pNFS helps in minimizing the NFS RPC overhead for large sequential read and write operations. The larger the write size, the better the performance. All this is techno-speak for pNFS providing performance boosts for the application and the workload.
If all this is so beneficial, why hasn’t this been adopted and implemented at a large scale? It turns out that the end users would love nothing but being able to deploy pNFS had it not been for the dependency of the ISV and EDA providers making their releases compatible with Red Hat Enterprise Linux® RHEL 6.4, a prerequisite for pNFS/NFS v4.1. Given that many applications need to be compatible with RHEL 6.4 and not all companies have dedicated hardware deployed by application or workload, the default setup reduces to the lowest common denominator version that supports all those apps. This leads to causing many companies to run their engineering farms in a suboptimum scenario, and only through a concerted effort and drive by all stakeholders will the engineering community benefit from the inherent enhancements of the parallel world. The message is: Ask your EDA/ISV provider to have their latest application releases support RHEL 6.4 and beyond with many benefits for all stakeholders: the application provider, the hardware administrator, the infrastructure provider, and the engineering end user. For more details on the pNFS value proposition, refer to this NetApp blog and do note that pNFS might not provide benefits for all workloads but is meaningful enough for many phases of the design such as verification that it warrants being deployed for greater overall benefits.
View attachment 9013
The NetApp optimized EDA workflow solution will be showcased at this week’s ARM TechCon, NetApp booth 707, which kicks off Wednesday October 30 at the Santa Clara Convention Center.
A typical EDA workflow consists of users submitting jobs through job schedulers that distribute tasks for execution to compute nodes that interact with (increasingly clustered) storage systems in order to complete the intended tasks in the most economical and expedient way.
View attachment 9012
The design steps can be typically grouped into four main phases:
1. Architect from a concept or specification a high-level structure that identifies functional blocks and how they interact.
2. Design and verify the functionality and timing performance of the intended features.
3. Implement and realize the design in its physical structure and connectivity and re-verify that the timing can still be met.
4. Manufacture the completed design that has been transferred (taped out).
In each phase of the design steps, the workload profile varies in the amount of random or sequential reads, writes, or mix and intensity of data versus metadata (that is, data about the data). The analysis (characterization) of this workload can provide interesting insight into how storage and file system versions (such as NFSv3, NFSv4.1/pNFS) affect the performance of the jobs.
Network file systems (such as NFSv3, common in EDA environments) traditionally have the metadata and data share the same I/O path. In a pNFS protocol (a minor revision of NFS4.1) the metadata is separate from the data path. Every node in the clustered Data ONTAP® environment is a metadata and a data node for a pNFS client. In clustered storage architectures such as NetApp® clustered Data ONTAP 8.2 with pNFS support, data paths are local to all nodes. Data volumes can be moved non-disruptively, and metadata traffic can be load balanced.
This is how data access occurs: pNFS clients are directed to the physical node that hosts a specific file. Data access over pNFS is therefore always direct. If a volume is moved from one physical node to another, the pNFS clients will send I/O requests to the new location.
Taking the example of a simulation workload, pNFS helps in minimizing the NFS RPC overhead for large sequential read and write operations. The larger the write size, the better the performance. All this is techno-speak for pNFS providing performance boosts for the application and the workload.
If all this is so beneficial, why hasn’t this been adopted and implemented at a large scale? It turns out that the end users would love nothing but being able to deploy pNFS had it not been for the dependency of the ISV and EDA providers making their releases compatible with Red Hat Enterprise Linux® RHEL 6.4, a prerequisite for pNFS/NFS v4.1. Given that many applications need to be compatible with RHEL 6.4 and not all companies have dedicated hardware deployed by application or workload, the default setup reduces to the lowest common denominator version that supports all those apps. This leads to causing many companies to run their engineering farms in a suboptimum scenario, and only through a concerted effort and drive by all stakeholders will the engineering community benefit from the inherent enhancements of the parallel world. The message is: Ask your EDA/ISV provider to have their latest application releases support RHEL 6.4 and beyond with many benefits for all stakeholders: the application provider, the hardware administrator, the infrastructure provider, and the engineering end user. For more details on the pNFS value proposition, refer to this NetApp blog and do note that pNFS might not provide benefits for all workloads but is meaningful enough for many phases of the design such as verification that it warrants being deployed for greater overall benefits.
View attachment 9013
The NetApp optimized EDA workflow solution will be showcased at this week’s ARM TechCon, NetApp booth 707, which kicks off Wednesday October 30 at the Santa Clara Convention Center.
Last edited by a moderator:
