Wiki

Clone wiki

computing / Home

This project provides an introduction and guide to using computing resources as part of the Biogeo lab.

Guides

Computing

  • Farm - College of Ag & Env Sci traditional compute cluster.
  • BigR - Large machine for R coding and processing.
  • Google Compute Engine - Configuring Google Compute Engine instances for Earth Engine use (Costs $)
  • Amazon - Compute and Storage options on the Amazon Cloud (Costs $)

Data Management

Read about the different Storage options, which are listed below.

General

  • SQLite - Working with a single file database.
  • R - Generic R tips

Data Center

  • NFS - Network File Storage,
    • Spatial02 - for big data and personal projects
    • Spatial03 - for ?
    • SSD3 - for serving data to the web
    • SSD - for tile caching and other web schemes
  • Hadoop - Distributed network file storage.

Cloud

  • Amazon - S3 data storage for use on Amazon Cloud or Public serving
  • Box - Box.com storage, manual batch file transfer
  • Google-Drive - Working with the Google Drive, especially from Linux
    • Try to avoid putting code in google-drive, especially when collaborating with multiple people. For that use Version Control (below)

Code Management

When in doubt do not put data into version control, in particular large binary files (pdf, tif, shp, etc)

Resources

Lab Notes

Discussion

  • Bigdata - Options for handling big data.

Get Help

Communication

Updated