Principal AI Network Engineer - Highrise AI

Remote·Posted today

cryptoweb3blockchainllmpythonnode

<div class="content-intro">   </div>About Highrise AI  Highrise AI is a GPU-native infrastructure platform built for modern AI workloads. Designed for developers and enterprises operating in performance-critical and security-sensitive environments, Highrise delivers scalable GPU compute across dedicated clusters, managed cloud, and confidential environments.  Highrise combines bare-metal performance with full-stack orchestration, API-driven access, and hardware-enforced isolation to support training, fine-tuning, and production-scale deployment of AI models. From reserved GPU capacity to enterprise-grade confidential compute, Highrise provides flexible infrastructure models tailored to the needs of builders operating at scale.  Beyond infrastructure, Highrise integrates disciplined operations, advanced automation, and on-site engineering expertise to sustain high uptime and predictable performance across deployments. These capabilities extend beyond Highrise-owned fleets: through managed services, Highrise also operates third-party GPU hardware on behalf of customers and partners, applying the same operational standards to infrastructure it does not own.  Highrise is a subsidiary of Hut 8 Corp., an energy infrastructure platform. Hut 8 integrates power, digital infrastructure, and compute at scale. Highrise serves as the compute and operations layer—designing, deploying, and operating next-generation GPU clusters from commissioning through production.  ROLE OVERVIEW  We are seeking a principal-level, hands-on AI Network Engineer to bring large-scale GPU cluster networks from physical installation and cabling to fully configured, validated, and commercially ready operation. Working alongside senior compute, storage, and infrastructure engineers, this person will lead network fabric bring-up, advanced configuration, performance tuning, and technical acceptance across AI training and supporting infrastructure networks.  Following deployment, the engineer will serve as a Tier 3 escalation point for complex production networking issues, driving root cause analysis, reliability improvements, and ongoing optimization of high-performance AI clusters.  KEY RESPONSIBILITIES  Network Fabric Bring-Up and Integration  <ul> <li data-leveltext="" data-font="Symbol" data-listid="23" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="1" data-aria-level="1">Lead network commissioning of materially complete AI clusters, including switch initialization, configuration validation, host connectivity, and readiness for production workloads. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="23" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="2" data-aria-level="1">Configure and integrate high-speed Ethernet and/or InfiniBand compute fabrics, including RDMA/RoCE, alongside storage, management, and out-of-band networks. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="23" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="3" data-aria-level="1">Validate topology, routing, redundancy, addressing, port mappings, optics, cabling, and interoperability across switches, NICs, and supporting systems. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="23" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="4" data-aria-level="1">Implement and refine network automation, configuration management, firmware/OS baselines, and repeatable fabric deployment procedures. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="23" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="5" data-aria-level="1">Partner with compute and storage engineers to resolve host-to-fabric integration issues and ensure end-to-end cluster readiness. </li> </ul> Performance Optimization and Acceptance  <ul> <li data-leveltext="" data-font="Symbol" data-listid="24" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="1" data-aria-level="1">Execute fabric qualification and acceptance testing for link health, error rates, bandwidth, latency, congestion, resilience, and scaling behavior. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="24" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="2" data-aria-level="1">Troubleshoot and tune RDMA transport, congestion control, QoS, PFC/ECN, routing, and switch/NIC settings as appropriate to the deployed fabric. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="24" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="3" data-aria-level="1">Analyze NCCL and distributed workload results with compute engineers to isolate communication bottlenecks and improve multi-node performance. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="24" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="4" data-aria-level="1">Use telemetry, switch counters, fabric management platforms, and diagnostic tools to identify degraded links, optical issues, packet loss, or unexpected congestion. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="24" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="5" data-aria-level="1">Own networking acceptance evidence, defect remediation, configuration sign-off, and readiness recommendations for commercial service. </li> </ul> Tier 3 Operations and Network Reliability  <ul> <li data-leveltext="" data-font="Symbol" data-listid="25" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="1" data-aria-level="1">Provide senior technical escalation for production incidents involving fabric instability, host connectivity, network performance, routing, or persistent hardware faults. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="25" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="2" data-aria-level="1">Lead deep troubleshooting and root cause analysis across switches, NICs, optics, cabling, and network operating systems; coordinate vendor engineering escalations. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="25" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="3" data-aria-level="1">Support network upgrades, expansions, failover testing, controlled configuration changes, and post-maintenance validation. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="25" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="4" data-aria-level="1">Develop diagnostic automation, monitoring standards, runbooks, and operational handoffs for NOC, SRE, and on-site technicians. </li> </ul> Cross-Functional Engineering and Delivery  <ul> <li data-leveltext="" data-font="Symbol" data-listid="26" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="1" data-aria-level="1">Collaborate with infrastructure engineering, OEMs, networking vendors, site operations, and customer-facing teams to meet deployment and service milestones. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="26" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="2" data-aria-level="1">Provide concise technical reporting on fabric health, acceptance results, outstanding defects, operational risks, and recommended improvements. </li> </ul> ABOUT YOU  Education and Experience  <ul> <li data-leveltext="" data-font="Symbol" data-listid="27" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="1" data-aria-level="1">Bachelor's degree in computer science, engineering, networking, or related discipline, or equivalent practical experience. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="27" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="2" data-aria-level="1">10+ years of hands-on data center or high-performance networking experience, including deep troubleshooting and technical leadership. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="27" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="3" data-aria-level="1">Demonstrated success deploying and operating large-scale, low-latency networks for AI, HPC, cloud, or similarly demanding distributed systems. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="27" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="4" data-aria-level="1">Strong expertise in Ethernet and/or InfiniBand fabrics, RDMA/RoCE, routing and switching, and multi-tier data center network architectures. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="27" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="5" data-aria-level="1">Experience with high-speed 200/400/800G networking, NICs, optical interconnects, performance testing, and network telemetry. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="27" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="6" data-aria-level="1">Proficiency with Linux networking and network automation using Python, Ansible, or equivalent tooling; familiarity with NVIDIA networking and fabric-management tools is highly desirable. </li> </ul> Core Competencies  <ul> <li data-leveltext="" data-font="Symbol" data-listid="28" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="1" data-aria-level="1">Expert-level problem solving across physical, link, transport, and application-facing network performance layers. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="28" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="2" data-aria-level="1">Strong ownership of deployment outcomes, acceptance criteria, operational reliability, and technical documentation. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="28" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="3" data-aria-level="1">Ability to make independent technical decisions while collaborating effectively with senior compute, storage, and systems engineers. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="28" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="4" data-aria-level="1">Clear communication with engineering leadership, customers, site teams, OEMs, and external network vendors. </li> </ul> ABOUT THE WORK ENVIRONMENT  This is a full-time, engineering role supporting large-scale AI compute infrastructure across multiple sites. Travel to data centers for cluster commissioning, acceptance, and complex troubleshooting will be required. Flexibility to assist with critical incidents or maintenance activities outside regular business hours is expected. This role is hybrid or remote depending on location. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.  WHAT MAKES HIGHRISE A GREAT PLACE TO WORK  At Highrise, you will directly influence the performance, reliability, and production readiness of large-scale AI clusters, work alongside experienced infrastructure engineers, and help define the technical standards for a growing global AI infrastructure platform.  At Highrise, you will have the opportunity to:  <ul> <li data-leveltext="" data-font="Symbol" data-listid="29" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="1" data-aria-level="1">Work with bright, driven peers from a range of educational and professional backgrounds including software development, energy, engineering, entrepreneurship, investment banking, private equity, and management consulting </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="29" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="2" data-aria-level="1">Design and pitch new products, services, and other initiatives to a leadership team consisting of serial entrepreneurs and seasoned executives and backed by a board of directors consisting of industry veterans of energy, finance, and government </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="29" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="3" data-aria-level="1">Debate ideas and alternatives in a truly meritocratic setting where the learning curve is steep and the lessons come from both senior and junior members of the team </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="29" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="4" data-aria-level="1">Build a lifelong network of friends and professional connections at the cutting-edge intersection of technology, energy, and infrastructure  </li> </ul>

Apply for this role