SGI 10-Gigabit Ethernet Network Adapter Instrukcja Użytkownika Strona 143
- Strona / 150
- Spis treści
- BOOKMARKI
Oceniono. / 5. Na podstawie oceny klientów
Hadoop: DFS IO Write Performance
• DFS IO included in Hadoop, measures sequential access throughput
• We have two map tasks each writing to a file of increasing size (1-10GB)
• Significant improvement with IPoIB, SDP and 10GigE
• With SSD, performance improvement is almost seven or eight fold!
• SSD benefits not seen without using high-performance interconnect!
• In-line with comment on Google keynote about I/O performance
Four Data Nodes
Using HDD and SSD
0
10
20
30
40
50
60
70
80
90
1
2
3
4
5
6
7
8
9
10
Average Write Throughput (MB/sec)
File Size(GB)
1GE with HDD
IGE with SSD
IPoIB with HDD
IPoIB with SSD
SDP with HDD
SDP with SSD
10GE-TOE with HDD
10GE-TOE with SSD
CCGrid '11
143
- A Tutorial at CCGrid ’11 1
- Computing Systems 3
- Cluster Computing Environment 4
- (http://www.top500.org) 5
- Grid Computing Environment 6
- Compute cluster 8
- Cloud Computing Environments 9
- Hadoop Architecture 10
- Memcached Architecture 11
- • Hardware components 13
- • Software components 13
- • Ex: TCP/IP, UDP/IP 14
- – E.g., PCI, PCI-X 15
- – One bit per wire 15
- – Not scalable: 15
- Myrinet (1993 -) 1 Gbit/sec 16
- IB Trade Association 19
- • Network speed bottlenecks 21
- • I/O interface bottlenecks 21
- (not shown) 24
- • Fast Messages (FM) 27
- • Myricom GM 27
- IB Hardware Acceleration 28
- • Interrupt Coalescing 29
- • Jumbo Frames 29
- • Hardware Checksum Engines 29
- TOE and iWARP Accelerators 30
- 2006 (v3.0), 2008 (v3.1) 34
- PCI-Express (PCIe) 34
- 2003 (Gen1), 2007 (Gen2) 34
- IB, HSE and their Convergence 36
- Traditional Ethernet 37
- IB Overview 38
- Components: Channel Adapters 39
- • Switches: intra-subnet 40
- – Not directly addressable 41
- IB Communication Model 43
- Queue Pair Model 44
- Memory Registration 45
- Memory Protection 46
- (Send/Receive Model) 47
- InfiniBand Device 48
- Hardware ACK 48
- Hardware Protocol Offload 51
- • Buffering and Flow Control 52
- • Switching and Multicast 52
- Buffering and Flow Control 53
- Virtual Lanes 54
- Service Levels and QoS 55
- Traffic Segregation Benefits 56
- Identifiers) 57
- Switch Complex 58
- – 3D Torus (Sandia Red Sky) 59
- More on Multipathing 60
- IB Multicast Example 61
- IB Transport Services 63
- Reliability 64
- Transport Layer Capabilities 65
- Data Segmentation 66
- Transaction Ordering 67
- Message-level Flow-Control 68
- Concepts in IB Management 71
- Subnet Manager 72
- HSE Overview 74
- Differences 75
- – Multi Stream Semantics 76
- Offloaded TCP 83
- Myrinet Express (MX) 85
- Datagram Bypass Layer (DBL) 86
- • Solarflare approach: 87
- Ethernet 90
- IB Network 90
- IB Verbs 90
- Application 90
- Hardware 90
- IB Transport 91
- IB iWARP/HSE RoE RoCE 92
- IB Hardware Products 94
- Tyan Thunder S2935 Board 95
- IB Hardware Products (contd.) 96
- – Nortel Networks 97
- • Support for VPI and RoCE 98
- – www.openfabrics.org 99
- – OFED 1.6 is underway 99
- Verbs Interface 100
- (libibverbs) 100
- • Within the hardware: 101
- OpenFabrics Software Stack 102
- InfiniBand in the Top500 103
- Cray Interconnect 104
- NUMAlink 104
- SP Switch 104
- CCGrid '11 106
- • Other small clusters 107
- • Integrated Systems 107
- Other HSE Installations 108
- Presentation Overview 109
- 1/10 GigE 110
- InfiniBand 110
- Case Studies 111
- Latency (us) 112
- Message Size (bytes) 112
- Bandwidth (MBps) 113
- MVAPICH/MVAPICH2 Software 115
- One-way Latency: MPI over IB 116
- Bandwidth: MPI over IB 117
- Bandwidth: MPI over iWARP 119
- Convergent Technologies: 121
- Traditional Model 123
- Possible SDP Model 123
- InfiniBand CA 123
- SDP vs. IPoIB (IB QDR) 124
- IB on the WAN 126
- Features 127
- • Supports SONET OC-192 or 128
- “appear” local 128
- Sunnyvale 129
- 700 miles 130
- 3300 miles 4300 miles 130
- Cluster B 131
- Communication Options in Grid 132
- Interface 133
- Modern WAN 133
- Performance of Memory Based 134
- Data Transfer 134
- ADTS-64MB 135
- IPoIB-64MB 135
- Application Level Performance 136
- Memcached Design Using Verbs 139
- Memcached Get Latency 140
- Memcached Get TPS 141
- Bandwidth with C version 142
- Hadoop Sort Benchmark 145
- Concluding Remarks 147
- Funding Acknowledgments 148
- Personnel Acknowledgments 149
- Web Pointers 150
Powiązane produkty i podręczniki dla Networking SGI 10-Gigabit Ethernet Network Adapter
(34 strony)© 2020, manymanuals.pl. Wszelkie prawa zastrzeżone. | 0.035 s |
Manymanuals.com
Manymanuals.de
Manymanuals.fr
Manymanuals.it
Manymanuals.pl
Manymanuals.cz
Manymanuals.es
Manymanuals-pt.com
Komentarze do niniejszej Instrukcji