Principal AI Infrastructure Engineer - Highrise AI

Remote·Posted today

cryptoweb3blockchainllmpythonnodekubernetes

<div class="content-intro">   </div>About Highrise AI  Highrise AI is a GPU-native infrastructure platform built for modern AI workloads. Designed for developers and enterprises operating in performance-critical and security-sensitive environments, Highrise delivers scalable GPU compute across dedicated clusters, managed cloud, and confidential environments.  Highrise combines bare-metal performance with full-stack orchestration, API-driven access, and hardware-enforced isolation to support training, fine-tuning, and production-scale deployment of AI models. From reserved GPU capacity to enterprise-grade confidential compute, Highrise provides flexible infrastructure models tailored to the needs of builders operating at scale.  Beyond infrastructure, Highrise integrates disciplined operations, advanced automation, and on-site engineering expertise to sustain high uptime and predictable performance across deployments. These capabilities extend beyond Highrise-owned fleets: through managed services, Highrise also operates third-party GPU hardware on behalf of customers and partners, applying the same operational standards to infrastructure it does not own.  Highrise is a subsidiary of Hut 8 Corp., an energy infrastructure platform. Hut 8 integrates power, digital infrastructure, and compute at scale. Highrise serves as the compute and operations layer—designing, deploying, and operating next-generation GPU clusters from commissioning through production.  ROLE OVERVIEW  We are seeking a senior, hands-on AI Cluster Integration Engineer to bring large-scale GPU compute clusters from physical installation to validated, commercially ready production. Working alongside a team of senior infrastructure engineers, this person will own critical elements of cluster bring-up, bare-metal provisioning, advanced systems configuration, performance optimization, and acceptance testing.  After deployment, the engineer will provide Tier 3 technical escalation and sustained performance support for operating clusters. The focus is on compute-platform readiness and workload performance rather than primary ownership of network switch and fabric configuration.  KEY RESPONSIBILITIES  Cluster Bring-Up and Systems Integration  <ul> <li data-leveltext="" data-font="Symbol" data-listid="12" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="1" data-aria-level="1">Lead technical bring-up of materially complete, racked, powered, and cabled AI clusters through production readiness and customer handoff. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="12" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="2" data-aria-level="1">Establish and validate system baselines across BIOS/BMC, firmware, GPU drivers, operating systems, and cluster software stacks. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="12" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="3" data-aria-level="1">Deploy and optimize automated bare-metal provisioning, imaging, node configuration, and lifecycle workflows at scale. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="12" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="4" data-aria-level="1">Configure and troubleshoot advanced Linux and GPU systems, including kernel, driver, accelerator, and host-level performance settings. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="12" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="5" data-aria-level="1">Integrate and validate Kubernetes and Slurm environments in partnership with platform, networking, and operations engineers. </li> </ul> Performance Engineering and Acceptance  <ul> <li data-leveltext="" data-font="Symbol" data-listid="13" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="1" data-aria-level="1">Execute NVIDIA Field Diagnostics, L11/L12 validation, burn-in, stability, and hardware health qualification procedures. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="13" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="2" data-aria-level="1">Run and analyze NCCL collective communication tests, GPU benchmarks, and distributed workload performance at node, rack, and cluster scale. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="13" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="3" data-aria-level="1">Troubleshoot GPU, NVLink/NVSwitch, PCIe, RDMA, and InfiniBand/Ethernet performance issues, partnering with network engineers on fabric-related remediation. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="13" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="4" data-aria-level="1">Identify configuration bottlenecks and optimize throughput, latency, scaling efficiency, and cluster reliability. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="13" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="5" data-aria-level="1">Own technical acceptance criteria, test evidence, defect tracking, remediation verification, and readiness recommendations for commercial service. </li> </ul> Tier 3 Operations and Reliability  <ul> <li data-leveltext="" data-font="Symbol" data-listid="14" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="1" data-aria-level="1">Serve as a senior escalation point for complex production incidents involving compute nodes, GPU software, provisioning, scheduling, and degraded performance. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="14" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="2" data-aria-level="1">Perform deep root cause analysis, coordinate vendor engineering escalations, and lead corrective actions for recurring or systemic issues. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="14" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="3" data-aria-level="1">Support fleet-level upgrades, configuration changes, cluster expansions, and post-maintenance validation with appropriate change controls. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="14" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="4" data-aria-level="1">Develop diagnostic tooling, automation, runbooks, and operational handoff material for SRE, NOC, and site support teams. </li> </ul> Cross-Functional Execution  <ul> <li data-leveltext="" data-font="Symbol" data-listid="15" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="1" data-aria-level="1">Work closely with infrastructure engineering, site operations, OEMs, and software/platform teams to resolve cross-stack issues and meet delivery milestones. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="15" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="2" data-aria-level="1">Provide concise technical status, benchmark findings, risks, and acceptance recommendations to engineering leadership and stakeholders. </li> </ul> ABOUT YOU  Education and Experience  <ul> <li data-leveltext="" data-font="Symbol" data-listid="10" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="1" data-aria-level="1">Bachelor's degree in computer science, engineering, or a related field, or equivalent practical experience. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="10" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="2" data-aria-level="1">8+ years of hands-on Linux systems engineering, HPC, cloud, or large-scale infrastructure experience, including senior-level troubleshooting. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="10" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="3" data-aria-level="1">Proven experience bringing GPU or HPC clusters into production and qualifying systems against performance and reliability targets. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="10" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="4" data-aria-level="1">Strong expertise in Linux, automation (Python/Bash/Ansible), bare-metal provisioning, configuration management, and systems diagnostics. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="10" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="5" data-aria-level="1">Hands-on experience with NVIDIA GPU platforms, NCCL, Field Diagnostics, and multi-node performance testing. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="10" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="6" data-aria-level="1">Practical experience with Kubernetes and Slurm; familiarity with high-speed Ethernet, InfiniBand/RDMA, and parallel or distributed storage. </li> </ul> Core Competencies  <ul> <li data-leveltext="" data-font="Symbol" data-listid="11" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="1" data-aria-level="1">Deep technical curiosity and the ability to isolate difficult problems across hardware, firmware, operating systems, and distributed workloads. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="11" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="2" data-aria-level="1">Strong ownership of outcomes, with a systematic approach to testing, documentation, root cause analysis, and continuous improvement. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="11" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="3" data-aria-level="1">Comfortable collaborating with highly experienced peer engineers while driving independent technical decisions and execution. </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="11" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="4" data-aria-level="1">Clear communication with engineering leadership, vendors, site personnel, and customer-facing stakeholders. </li> </ul> ABOUT THE WORK ENVIRONMENT  This is a full-time, engineering role supporting large-scale AI compute infrastructure across multiple sites. Travel to data centers for cluster commissioning, acceptance, and complex troubleshooting will be required. Flexibility to assist with critical incidents or maintenance activities outside regular business hours is expected. This role is hybrid or remote depending on location. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.  WHAT MAKES HIGHRISE A GREAT PLACE TO WORK  At Highrise, you will directly influence the performance, reliability, and production readiness of large-scale AI clusters, work alongside experienced infrastructure engineers, and help define the technical standards for a growing global AI infrastructure platform.  At Highrise, you will have the opportunity to:  <ul> <li data-leveltext="" data-font="Symbol" data-listid="16" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="1" data-aria-level="1">Work with bright, driven peers from a range of educational and professional backgrounds including software development, energy, engineering, entrepreneurship, investment banking, private equity, and management consulting </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="16" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="2" data-aria-level="1">Design and pitch new products, services, and other initiatives to a leadership team consisting of serial entrepreneurs and seasoned executives and backed by a board of directors consisting of industry veterans of energy, finance, and government </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="16" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="3" data-aria-level="1">Debate ideas and alternatives in a truly meritocratic setting where the learning curve is steep and the lessons come from both senior and junior members of the team </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="16" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="4" data-aria-level="1">Build a lifelong network of friends and professional connections at the cutting-edge intersection of technology, energy, and infrastructure  </li> </ul>

Apply for this role