[Lablife] Lab and personal cloud for data management

I am currently managing two NAS (Network Attached Storage) systems, one for the lab and one personal, as shown in the picture below. When I joined the current lab for my PhD degree, I persuaded my professor to establish a Lab cloud. Today I’d like to discuss the reasons behind my proposal and why I use NAS for data management.

Reasons for persuasion and advantages of NAS implementation

1. Protection against sudden hard drive failure: In the past, my 4TB hard drive suddenly failed, requiring a $500 recovery cost. Implementing RAID protects data from unexpected hard drive failures.

2. Prevention of data loss due to device theft: A lab member lost his personal laptop, resulting in the loss of research data. Using NAS prevents data loss from portable device theft.

3. Centralized data management: A large amount of lab-generated data (NGS and proteomics-related) was scattered across multiple external hard drives without any organization. Centralizing data management through NAS ensures better data organization.

4. Unified data management with automatic synchronization/backup: Research-related data and files were managed separately on different computers with manual backups. NAS enables data management under a single system/rule through automatic synchronization and backup.

5. Real-time network-based data management: Data generated by experimental/analytical equipment at different locations was inconveniently transferred and managed via USB. NAS allows for real-time organization and management of files in personal folders via the network, simultaneously with the analysis.

6. Cost-effective handling of large data: Web-based solutions like Google Drive are secure but expensive and slow. NAS, although relatively less secure, is better suited for handling large amounts of data.

Choosing synology for lab and personal cloud setup

When selecting the NAS equipment necessary for setting up a lab cloud, hardware across manufacturers was similar, but significant price differences arose due to software differences. Synology offered optimized OS and applications, making it the preferred choice.

Important considerations during initial setup

• RAID Configuration 

  • To prevent data loss due to hard drive failure (set to RAID 5 to ensure safety even if one drive fails, at the cost of reduced usable capacity).

• Network and external access setting

  • Domain configuration
  • Port forwarding in Wi-Fi router
  • WebDAV server & RaiDrive setup

• Security settings

  • Issuing and registering SSL certificates for HTTPS access.
  • Reverse proxy setup
  • Two-Factor Authentication (2FA)
  • Firewall configuration
  • Automatic IP Blocking

• Internal data backup

  • Snapshot Replication
  • Regular backup scheduling to external storage or cloud (Limited by the need for additional storage/NAS)

• Power management

  • Wake-on-LAN (WOL) setup to allow remote power-on of NAS over the network, enabling convenient and energy-efficient management

  • UPS (Uninterruptible Power Supply) setup to prevent data loss during power failures (Limited by the need for UPS equipment)

• Creating user accounts

• Assigning different access permissions to each account

• Structuring folders according to purpose

Learning from a near disaster

While the equipment was in use, a sudden fire in the lab nearly resulted in the total loss of the NAS. This incident highlighted the vulnerability of relying on a single NAS and the necessity of data backup across devices located in different locations for complete security. If the budget allows, additional equipment will be set up.

Complementary use of NAS and Google Drive

Despite the many advantages of NAS, its stability has some weak points. Therefore, I use both Google Drive (2TB) and NAS simultaneously for different purposes.

Conclusion

My initial motivation was to enhance the convenience for lab members and prevent data loss, contributing to the lab’s overall progress. Although the first setup consumed significant time, it was a meaningful experience as I learned a lot during the process and broadened my perspective on future data management.

---

Personal NAS

A few years after setting up the first NAS, I installed a personal NAS to manage large volumes of data. Since it was my second setup, I completed it relatively quickly. Here are two significant changes I noted:

1. The DS923+ supports 10GbE data transfer speeds. This capability is remarkable as it enables the transfer of several gigabytes of data in just a few seconds, significantly reducing time. To take advantage of this feature, you need two items (10GbE Network Upgrade Module & Thunderbolt 10G Ethernet Adapter) and a LAN that supports 10GbE (CAT.6a or higher). I strongly recommend using this capability. Once configured, you can access the NAS via SMB on both Windows and Mac.

2. While the Lab cloud used 10TB NAS drives, I opted for 18TB drives for the personal NAS, given that single drives with up to 22TB capacity are now available. Considering the budget, I chose the 18TB drives. Despite the NAS being a 4-bay unit, the large capacity of the drives provided substantial storage. However, one downside is the increased noise level, likely due to the higher number of components per unit area.