Infiniband Protocol Training

Course Name
Infiniband Protocol Training

Course Overview
This course on Infiniband Protocol training provides the foundation for the InfiniBand technology from a usability point of view and builds from the details of the InfiniBand specification. It is designed to meet the needs of anyone who is involved in installing, configuring, managing, troubleshooting and monitoring InfiniBand fabrics.

Target Audience

  • Professionals and Students who are working in Storage/Networking/IT Domain
  • Developers, Testers/QA and Verification Engineers who are working on or keen to know Infiniband Protocol

Fee, Schedule & Registration
Click Here for Infiniband Protocol Training course training schedule, fee and registration information.

Infiniband Protocol Training Course Outline

Basic Terms and Concepts
InfiniBand Advantages
Definition of a Subnet
Packet Addressing Basics
Every Packet Contains a BTH
Channel Adapters
Role of Switches and Routers
Repeater’s Role
It’s All About Message Passing
Intro to Attributes and Managers
Definition of an Attribute
MAs Handle Access Requests
MA’s Response
Managers Use Special Packets Called MADs
Attribute Format and Documentation Conventions
QP: Message Transfer Mechanism
A QP Is a Bi-Directional Message Transport Engine
Verb Layer Is an OS-Independent API
QP Context Defines QP’s Operational Characteristics
Sending a Message to a Destination CA
Intro to Transport Types
Four IBA Transfer Protocol Flavors
Two Non-IBA Service Types
Intro to Send/Receive Operations
How Is a Message Transmitted or Received?
SQ Operation Types
RQ Operation Types
Division of Labor
Brief Layer Descriptions
Layers in CAs, Switches and Routers.
Physical Layer Overview
Link Layer Overview
Network Layer Overview
Transport Layer Overview
Subnet-Local Addressing
Port Numbering
LID Address Space
LID’s Purpose: Packet Routing Within Subnet
Assigning Port’s Base LID Address
Why Assign a LID Range to a Port?
Port’s Decode of DLID Address
Port’s Selection of SLID on Packet Transmit
SM Path Database
LID Rule Summary
Global Addressing
Global Routing: Source/Destination CAs in Different Subnets
IPv4 Addresses Too Limiting
IBA Global Address = IPv6 Address
IPv6 Addressing
Assignment of Port’s Subnet ID and GUID(s)
Intro to the Managers
The SM
General Services Managers
Subnet Administrator’s Role
Introduction to Event Notification
Intro to Connection Establishment
A CA Is a Provider of Services
Locating a Specific CA
Discovering Services a CA Provides
RC/UC Connection Establishment
RD Connection Establishment
UD Connection Issues
PSN Usage
Overall Size of PSN Range
Requester QP’s SQ Logic PSN Generation and Verification
Responder QP’s RQ Logic Request PSN Verification
QP Verbs and QP State Machine
QP-Related Verbs
The QP State Machine
QP Creation
Software Control of QP State
QP Setup Is Performed in a Defined Sequence
Reset State
Initialized State
Ready to Receive State
Ready to Send State
SQ Drain (SQD) State
SQ Error State
Error State
WRs, WQEs, and CQEs
Once Posted to SQ or RQ, WR Is Called a WQE
WQE Execution and Completion Order
RDMA Read Relaxed Ordering Rules
Completion Queues (CQs) and CQEs
Asynchronous Events and Errors
Registering a Handler
Affiliated Asynchronous Events
Affiliated Asynchronous Errors
Unaffiliated Asynchronous Errors
Memory Protection
The Problems and Solutions
Virtual-to-Physical Page Mapping Background
Memory Regions
Memory Windows
Protection Domains
Other Protection Mechanisms
The IBA Protection Mechanisms
Memory Access Protection (PD, L_Key, and R_Key)
PDs and UD Service
Partition Key (P_Key)
SM-Related Protection Mechanisms
RD Domain
Queue Key (Q_Key)
Baseboard Management Key (B_Key)
RC Transport Service
RC Support Requirement
RC Basic Operational Characteristics
RC Connection Establishment
Packet Opcodes
RC Message Transfer Primer
QP State before Any Messages Are Transferred
Standard Operation in Fast, Error-Free Environment
Traffic Reduction
Packet Delivery Delays
Packet Loss
Nak Errors
RQ Logic’s Error Detecting and Handling
End-to-End Flow Control
SQ Logic Can Use MSN to Complete WQEs
Additional Reference Information
UC Transport Service
UC Support Requirement
In RC, Responses Are Expected
UC Is a Subset of RC
UC Transport Service Type’s Basic Characteristics
Requester QP’s SQ Logic Operation
Responder QP’s RQ Logic Operation
RD Transport Service
RD Support Requirement
Many Similarities to RC
RD Basic Operational Characteristics
The Major Differences from RC
The Scheduler
Keep RDC Operational If a QP Goes Down
Additional Reference Material
UD Transport Service
UD Support Requirement
Maximum Message Length Is One PMTU
Basic Operational Characteristics
Messaging with the Desired Remote Service
Address Handles
PD Check Performed before Message Is Processed
SQ Logic Operation
RQ Logic Operation
Raw Transport Service Types
Goal: Tunneling Non-IBA Packets through IBA Network
Solution: Disguise It as Special-Purpose IBA Packet
Raw QPs Are Used to Transmit/Receive Non-IBA Packets
Raw QP Support Requirement
Raw Transport Services Are Unreliable
Send and Receive Are Only Supported Operation Types
Basic Operational Description
Raw Datagrams Do Not Have an ICRC
Raw Datagrams Do Not Have a Destination QP
LRH:LNH Indicates Packet Type
Raw IPv6 Datagrams
Raw EtherType Datagrams
Definition Of Multicasting
Only UD and Raw QPs Can Participate in Multicasting
UD Multicasting
Raw Packet Multicasting
Automatic Path Migration
Causes of a Path Migration
APM-related Elements
Normal Operation before APM Enabled
Enabling APM
Automatic Hardware Trigger of APM
Loading a New Path or a Tertiary Path
Additional Reference Information
Static Rate Control
How Fast Can a Port Transmit Packets?
Problems and Solutions
Detailed Description of the Link Layer
Link Layer Functional Overview
Link State Machine
Detailed Description of LRH
QoS within the Subnet: SL and VLs
Detailed Description of VL Arbitration
Link-Level Flow Control
Packet CRCs
Packet Receive State Machine
Data Packet Check
Link Packet (Flow Control Packet) Check
Switch Performs Packet Forwarding
Overview of Router Port’s Link Layer
Detailed Physical Layer Description
Module Basics, port types
Signal Naming Conventions
Electrical Signaling and Copper Cable
Link Layer to Physical Layer Interface
Transmit Logic Functions
Receiver Logic Functions
Link Training
Physical Layer Error Handling
Performance Counters
The SMI on a CA and Router
The SMI on Switches
Detailed Switch Handling of SMPs
Detailed CA or Router Handling of SMPs
SM Can Reside in a Switch
SMI Is a Privileged Resource
SMI Only Communicates with Other SMIs
Port States SMPs Can Be Sent and Received In
SMPs Never Leave the Subnet
Setting Up an HCA Port’s SMI
How the SM Sends a Message and Handles a Response
SMP Source and Destination
The SMI and the Q_Key
The SMI and Partitions
Detailed Description of MADs
Software Times Return of MAD Response
Basic MAD Contents
Event Subscription and Event Forwarding
The Notice Queue
SM Methods and Attributes
SM MAD Formats, Methods, Attributes and Traps
SMA Notice Support
Multiple SMs
Multi-Vendor SM Failover Is Not Supported
A Subnet Can Have More Than One SM
Introduction to the SM States
SM Control Packets
Multiple Master SMs
How Packets Are Normally Routed
Packet Routing During Discovery
Scenario: Sweep at Startup
Accessing Device along Partially Configured Path
The GSMs, GSAs, and GSIs
Required/Optional GSAs
The SA and the GSI
GMPs Can Transit Routers
QP1 Is a Controlled-Access QP
P_Key Insertion and Checking
Detailed Description of SA
SA Accessed Using GMPs
Location of the SA
Requester Access Authorization
SA Methods and Attributes
Record Identifier (RID) Definition
Fetch Entire Database
Reliable Multi-Packet Transaction Protocol
Baseboard Management
Chassis and Module
Chassis Baseboard Management Elements
Passively Managed Chassis
Module BM Elements
Non-Module IBA Devices
BM Sending a Command to the MME
CME Sends a Command to the BM
BM-related Traps
Performance Management
The Role of Performance Management (PM)
Required Features
Optional Features
Performance Management MAD Format
Performance Methods
Mandatory Performance Attributes
Optional PM Attributes
Communications Management
Definition of Client and Server
Definition of Active and Passive CM
Three Models Are Supported
Stale Communications Channel
Device Management
Definition of IOU and IOCs
DM MAD Format, Methods and Attribute

Subscribe to our Newsletters (Subject-wise). Participate in the Sanfoundry Certification contest to get free Certificate of Merit. Join our social networks below and stay updated with latest contests, videos, internships and jobs!

Youtube | Telegram | LinkedIn | Instagram | Facebook | Twitter | Pinterest
Manish Bhojasia - Founder & CTO at Sanfoundry
Manish Bhojasia, a technology veteran with 20+ years @ Cisco & Wipro, is Founder and CTO at Sanfoundry. He lives in Bangalore, and focuses on development of Linux Kernel, SAN Technologies, Advanced C, Data Structures & Alogrithms. Stay connected with him at LinkedIn.

Subscribe to his free Masterclasses at Youtube & discussions at Telegram SanfoundryClasses.