Source

icis-largedata2011-yt / index.html

Full commit
<!DOCTYPE html>

<!--
  Google HTML5 slide template

  Authors: Luke Mahé (code)
           Marcin Wichary (code and design)
           
           Dominic Mazzoni (browser compatibility)
           Charles Chen (ChromeVox support)

  URL: http://code.google.com/p/html5slides/
-->

<html>
  <head>
    <title>Viz Capability</title>

    <meta charset='utf-8'>
    <!--<script src='http://html5slides.googlecode.com/svn/trunk/slides.js'></script>-->
    <script src='slides.js'></script>
  </head>
  
  <style>
    /* Your individual styles here, or just use inline styles if that’s
       what you want. */
    
    
  </style>

  <body style='display: none'>

    <section class='slides layout-regular template-default'>
      
      <!-- Your slides (<article>s) go here. Delete or comment out the
           slides below. -->
        
        
      
      <article>
        <h1>
          Astrophysically-Aware Data Handling
        </h1>
        <h2>
          (<span style="font-family: Inconsolata;">yt</span>: a library for simulation analysis)
        </h2>
        <p>
          Matthew Turk
          <br>
          August 11, 2011
        </p>
      </article>
      <article>
        <ul>
          <li><a style="font-family:Inconsolata;" href="http://yt.enzotools.org/">yt.enzotools.org</a></li>
          <li>Python, Cython, C, C++</li>
          <li>Parallelized</li>
          <li>Works with multiple codes</li>
          <li>20 or so developers, ~100 (generously) users</li>
          <li>100% FLOSS component stack</li>
          <li>Focused on <i>pragmatic</i> data access</li>
          <li><span style="font-family:Inconsolata;">astro-ph/1011.3514</span>
        </ul>
        <h2>Factsheet</h2>
      </article>
      <article>
        <iframe
          src="http://hg.enzotools.org/yt/wiki/CodeSupportLevels"></iframe>
      </article>
      <article>
        <ol class="build">
          <li>Data I/O from disk</li>
          <li>Coherent data structures</li>
          <li>Fundamental operations</li>
          <li>Canned analysis and viz tasks</li>
        </ol>
        <h2>Component architecture for pipeline construction.</h2>
      </article>
      <article>
        <qnq>
          Perform rich analysis on data, independent of source.
        </qnq>
      </article>
      <article>
        <ol class="build">
          <li>Scripting interface</li>
          <li>Small, targeted GUIs</li>
          <li>Web utilities</li>
          <li>ParaView plugin</li>
        </ol>
        <h2>API-driven development encourages multiple access points</h2>
      </article>
      <article>
        <pre style="font-family: Inconsolata;">from yt.mods import *
import yt.visualization.volume_rendering.camera as camera
pf = load("RedshiftOutput0010.dir/RedshiftOutput0010")
cam = camera.HEALpixCamera([0.5, 0.5, 0.5], 50./pf['mpc'], 
nside = 64, pf = pf,
log_fields = [False])
bitmap = cam.snapshot("allsky.png") </pre>
      <center><img src="allsky.png" height=300></center>
      </article>
      <article>
        <center><img src="reason_ss.png" width="800"/></center>
      </article>
      <article>
        <iframe src="http://localhost:7000" frameborder="0" allowfullscreen>
        </iframe>
      </article>
      <article>
        <iframe width="425" height="349"
          src="http://www.youtube.com/embed/cOv4Ob2q1fM" frameborder="0"
          allowfullscreen></iframe>
        <h2>API-driven development encourages multiple access points</h2>
      </article>
      <article>
        <qnq>Set up systems of objects and query them for data.</qnq>
      </article>
      <article>
        <ul>
          <li>Profiles (1-, 2-, 3-D)</li>
          <li>Projections (on- and off-axis)</li>
          <li>Querying objects for data</li>
          <li>SEDs and PopSyn</li>
          <li>Volume rendering</li>
          <li>Halo finding</li>
          <li>Light-cones</li>
          <li>Pencil beams</li>
        </ul>
        <h2>Some "canned" analysis</h2>
      </article>
      <article>
        <pre style="font-family: Inconsolata;">@derived_field
def Dinosaurs(field, data):
            return data["BioMass"] * data["ExtinctionConstant"]</pre>
        <h2>Transparently derived fields</h2>
      </article>
      <article>
        <pre style="font-family: Inconsolata;">@derived_field
def DivV(field, data):
    ds = 2.0 * data['dx'].flat[0]
    f  = data["x-velocity"][2:,1:-1,1:-1]/ds
    f -= data["x-velocity"][:-2 ,1:-1,1:-1]/ds
    ds = 2.0 * data['dy'].flat[0]
    f += data["y-velocity"][1:-1,2:,1:-1]/ds
    f -= data["y-velocity"][1:-1,:-2 ,1:-1]/ds
    ds = 2.0 * data['dz'].flat[0]
    f += data["z-velocity"][1:-1,1:-1,2:]/ds
    f -= data["z-velocity"][1:-1,1:-1,:-2 ]/ds
    new_field = na.zeros(data["x-velocity"].shape, dtype='float64')
    new_field[1:-1,1:-1,1:-1] = f
    return new_field</pre>
        <h2>Transparently derived fields</h2>
      </article>
      <article>
        <qnq>From components, pipelines can be constructed</qnq>
      </article>
      <article class="smaller">
      <pre style="font-family: Inconsolata;">from yt.mods import *

fn = "RedshiftOutput0005" # parameter file to load
pf = load(fn) # load data

# First we run our halo finder to identify all the halos in the dataset.  This
# can take arguments, but the default are pretty sane.
halos = HaloFinder(pf)

f = open("%s_halo_info.txt" % pf, "w")

# Now, for every halo, we get the baryon data and examine it.
for halo in halos:
    # The halo has a property called 'get_sphere' that obtains a sphere
    # centered on the point of maximum density (or the center of mass, if that
    # argument is supplied) and with the radius the maximum particle radius of
    # that halo.
    sphere = halo.get_sphere()
    # We use the quantities[] method to get the total mass in baryons and in
    # particles.
    baryon_mass, particle_mass = sphere.quantities["TotalQuantity"](
            ["CellMassMsun", "ParticleMassMsun"], lazy_reader=True)
    # Now we print out this information, along with the ID.
    f.write("Total mass in HOP group %s is %0.5e (gas = %0.5e / particles = %0.5e)\n" % \
            (halo.id, baryon_mass + particle_mass, baryon_mass, particle_mass))
f.close()</pre>
      </article>
      <article>
        <h3>It's Documented!</h3>
        <iframe src="http://yt.enzotools.org/doc/" width="100%"></iframe>
      </article>
      <article>
        <center><img src="Reion512_600px.png" height="100%"/></center>
        <h3>Wise &amp; Cen</h3>
      </article>
      <article>
        <center><img src="halo_0029.png" height="100%"/></center>
        <h3>Hummels &amp; Bryan</h3>
      </article>
      <article>
        <center><img src="plot.png" width="100%"/></center>
        <h3>Turk &amp; Abel</h3>
      </article>
      <article>
        <center><img src="PopIII_5000AU.png" height="100%"/></center>
        <h3>Turk &amp; Oishi</h3>
      </article>
      <article>
        <center><img src="data0024_clump_volrend.png" height="100%"/></center>
        <h3>Lee, Chang &amp; Murray</h3>
      </article>
      <article>
      <center><object type="application/x-shockwave-flash" width="640" height="540" data="http://vimeo.com/hubnut/?user_id=user3192834&amp;color=FFFFFF&amp;background=000000&amp;fullscreen=1&amp;slideshow=1&amp;stream=group&amp;id=34915&amp;server=vimeo.com"> <param name="quality" value="best" />       <param name="allowfullscreen" value="true" />     <param name="allowscriptaccess" value="always" /> <param name="scale" value="showAll" /> <param name="movie" value="http://vimeo.com/hubnut/?user_id=user3192834&amp;color=FFFFFF&amp;background=000000&amp;fullscreen=1&amp;slideshow=1&amp;stream=group&amp;id=34915&amp;server=vimeo.com" /></object> </center>
      </article>
      <article>
        <h2>Room for Growth</h2>
        <ul>
          <li>Not great with pure N-body datasets</li>
          <li>Octree support needs improvement</li>
          <li>Still growing our developer community!</li>
          <li>Performance optimization</li>
        </ul>
      </article>
      <article>
        <h2>Future Directions</h2>
        <ul>
          <li>Further, multi-level parallelism</li>
          <li>Workshop/conference possible in 2012</li>
          <li>Inline physics modules and evolution</li>
          <li>Greater integration as an "astrophysics library"</li>
          <li>Better support for particles and octree codes</li>
          <li>Closer integration with Planetaria output formats</li>
        </ul>
      </article>
      <article>
        <h2>Thank you.</h2>
      </article>
    </section>
  </body>
</html>