
Glenn K. Lockwood, Ph.D.
HPC Systems Architect
National Energy Research Scientific Computing Center
Lawrence Berkeley National Laboratory
/
I am a high-performance computing systems architect at the National Energy Research Scientific Computing Center (NERSC) at Lawrence Berkeley National Laboratory. NERSC provides high-performance computing resources for thousands of researchers around the world each year, and my role is to ensure that our storage systems balance advanced technologies with production reliability to enable science that can be done nowhere else. To this end, my work focuses on I/O performance analysis, extreme-scale storage architectures, and emerging I/O technologies and interfaces.
I have worked in the genomics industry in the past, and my formal education is in ceramic engineering. More information about my current and past work can be found in my CV and on LinkedIn, and excruciating details about my life are in my biography.
This site contains various notes I've taken related to technical computing over
the years. I welcome any feedback about its contents at the e-mail address
listed at the bottom of every page. You can also find me on Twitter and in
the #hpc
channel on freenode.
Data-Intensive Computing
The following pages document some of the different faces of data-intensive scientific computing in which I've worked over the last few years. I haven't used Hadoop or R since I left the San Diego Supercomputer Center in 2014, so those guides may seem a bit dated now.
Practical Data Analysis
- Implementing the Simplest Neural Network
- Implementing a Multilayer Neural Network
- Dask in Practice
- Working with Jupyter and Matplotlib
Topics in Storage
- I/O Forwarding for Extreme-Scale Systems
- Lustre Pearls of Wisdom
- Principles of Non-volatile RAM
- Principles of Object Storage
- Blog: Reviewing the state of the art of burst buffers
- Blog: VAST Data's storage system architecture
Topics in Hadoop
- Conceptual Overview of Map-Reduce and Hadoop
- Parsing VCF Files with Hadoop Streaming
- Running Hadoop on HPC Clusters
- Writing Hadoop Applications in Python with Hadoop Streaming
Parallel Computing with R
- Using R on HPC Clusters
- Parallel Options for R
- lapply-based Parallelism
- foreach-based Parallelism
- Alternative Forms of Parallelism
- Map-reduce-based Parallelism
HPC Howtos
In the past I have worked as both a user services consultant and a systems engineer. This section contains a few notes relevant to using and managing HPC resources, but the bulk of my HPC how-to guides remains hosted at the San Diego Supercomputer Center.
- Building and Managing Darshan
- Getting started with the NVIDIA Jetson Nano
- Hacking Darshan
- Managing Process Affinity in Linux
- Blog: Building IPM 0.983 for Lightweight MPI Profiling
- Blog: Quantum ESPRESSO: Compiling and Choice of Libraries
- Blog: Quick MPI Cluster Setup on Amazon EC2
Performance & Benchmarks
This section contains a number of disjoint pieces revolving around performance analysis.
- Building and Running IO500
- Getting Started with elbencho
- Historical Performance Benchmarks
- Parallel I/O Benchmarks
- Power Consumption
- Blog: Basics of I/O Benchmarking
- Blog: Dual-rail QDR as an Alternative to FDR InfiniBand
- Blog: Perl vs. Python Performance
- Blog: Quantum ESPRESSO: Performance Benefits of Vendor-Optimized Libraries
- Blog: SR-IOV with InfiniBand Performance
Materials Science
When I was a student I maintained a variety of notes about different topics relevant to ceramic engineering, in which I got my undergraduate degree. I've re-uploaded them here for posterity.
- Molecular Dynamics Tutorial
- Glass Compositions
- Overview of Alumina
- Overview of Glasses
- Overview of Metals
- Overview of Silicon Carbide
- Overview of Silicon Nitride
- Overview of Zirconia
- Statistical Design using Orthogonal Arrays
Miscellany
I've created the following pages about my hobbies mostly for my own entertainment. They have little to do with what I do professionally, but they might be of interest to people who hold similar interests.
Electronics
- Experimenting with digipots and ADCs
- Programming the BMC2835 SOC
- Programming the HD44780 LCD Display with Raspberry Pi
- Understanding bipolar junction transistors
- Understanding the 555 Timer IC
- Understanding the TWSU Synth Kit
- Using the MAX7219CNG LED Display Driver
- Blog: Learning electronics with roulette, datasheets, and Raspberry Pi
Sysadmin Howtos
- Configuring Raspberry Pi as a Wireless-to-Wired Ethernet Island
- Configuring Raspberry Pi for Headless Boot
- Configuring a Solaris 10 Server
- Configuring dmake
- Converting Disk Block Size on Solaris
- Getting started with the Terasic DE10-Nano