Where in the software stack should we encrypt data? My answer is, do encryption as low in the storage stack where it works well. The storage stack In particular, encrypted data does not compress well. Compression works by looking for and removing redundancy from the data. Effective encryption hides patterns and redundancies in the data, … Continue reading Where to do it? Encryption
Where to do it? Compression
Where in the software stack should we compress data? My answer is, do compression as high in the storage stack where it works well. The storage stack In other words, if an application data can do its own compression, let it compress. Compression relies on reducing redundancies in the data set - if something can … Continue reading Where to do it? Compression
Where to do it? Layers in the storage stack
Storage operations, such as compression, tiering, and replication, may happen throughout the "storage stack", and in some cases the same general operation could happen in more than one layer of the stack. When there is a choice, choosing correctly can make a big difference in performance and scalability. But before I talk about these different … Continue reading Where to do it? Layers in the storage stack
Storage metrics: Bandwidth, “IOPs”, and Latency
Typically when I talk with clients about storage metrics for performance, they will typically focus on either bandwidth or "IOPs". But really there are three dimensions of storage performance! Let's consider these three storage performance metrics and how we can design systems to work around shortcomings. First, there is bandwidth (or throughput). This is a … Continue reading Storage metrics: Bandwidth, “IOPs”, and Latency
Best Practices for building a stretched cluster with Spectrum Scale
The Spectrum Scale stretched cluster architecture is a highly-available parallel file system "stretched" across two data centers or sites. We call this an active-active configuration, because applications may access the same data sets from either site, concurrently, and enjoy file system consistency. You can learn about this architecture from the Expert Talk podcast I gave … Continue reading Best Practices for building a stretched cluster with Spectrum Scale
Getting started with Spectrum Scale Developer Edition – Downloading
Previously I wrote about IBM's Spectrum Scale Developer Edition. To get the code, you start by navigating to https://www.ibm.com/products/scale-out-file-and-object-storage. Spectrum Scale product page Click on the "Try free developer edition" button near the top. This will lead you to a form to either create an IBM account or log in using your existing "IBMid": If … Continue reading Getting started with Spectrum Scale Developer Edition – Downloading
IBM Spectrum Scale Developer Edition
IBM Spectrum Scale is a flexible parallel file system, but that flexibility comes at the price of some complexity. The well-designed command line interface and product consistency go a long way to mitigate that complexity, but it still takes time to master a technology as sophisticated as a parallel file system. Wouldn't it be nice … Continue reading IBM Spectrum Scale Developer Edition
Controlling what Spectrum Scale file systems are mounted on a node
When a node starts running Spectrum Scale, the -A attribute of each Spectrum Scale file system is checked to determine whether or not the file system should be mounted. File systems configured with -A yes will automatically be mounted when the node starts Spectrum Scale. File systems configured with -A automount will be mounted when the file … Continue reading Controlling what Spectrum Scale file systems are mounted on a node
Types of networks in Spectrum Scale
Spectrum Scale uses several logical networks for its communication.We use the term logical network to make clear that these are functions that may or may not use different physical networks, depending on configuration. Spectrum Scale daemon and administrative networks Daemon interface and network The Spectrum Scale daemon, mmfsd, runs on every active Spectrum Scale node, … Continue reading Types of networks in Spectrum Scale
Disaster Recovery planning and Spectrum Scale
There are multiple ways to use Spectrum Scale to prepare for disaster and help with recovery. An application is said to be active/passive if it runs only on a primary site with a particular data set, and only runs on a secondary site with that data set if the primary site is down. Active/passive configurations … Continue reading Disaster Recovery planning and Spectrum Scale
Parallel File Systems 