julio lopez - Academia.edu (original) (raw)
Uploads
Papers by julio lopez
2008 3rd Petascale Data Storage Workshop, 2008
In this work we present an scientific application that has been given a Hadoop MapReduce implemen... more In this work we present an scientific application that has been given a Hadoop MapReduce implementation. We also discuss other scientific fields of supercomputing that could benefit from a MapReduce implementation. We recognize in this work that Hadoop has potential benefit for more applications than simply datamining, but that it is not a panacea for all data intensive applications. We provide an example of how the halo finding application, when applied to large astrophysics datasets, benefits from the model of the Hadoop architecture. The halo finding application uses a friends of friends algorithm to quickly cluster together large sets of particles to output files which a visualization software can interpret. The current implementation requires that large datasets be moved from storage to computation resources for every simulation of astronomy data. Our Hadoop implementation allows for an in-place halo finding application on the datasets, which removes the time consuming process of tranferring data between resources.
Monthly Notices of the Royal Astronomical Society, 2012
We examine predictions for the quasar luminosity functions (QLF) and quasar clustering at high re... more We examine predictions for the quasar luminosity functions (QLF) and quasar clustering at high redshift (z 4.75) using MassiveBlack, our new hydrodynamic cosmological simulation which includes a self-consistent model for black hole growth and feedback. We show that the model reproduces the Sloan QLF within observational constraints at z 5. We find that the high-z QLF is consistent with a redshiftindependent occupation distribution of BHs among dark matter halos (which we provide) such that the evolution of the QLF follows that of the halo mass function. The sole exception is the bright-end at z = 6 and 7, where BHs in high-mass halos tend to be unusually bright due to extended periods of Eddington growth caused by high density cold flows into the halo center. We further use these luminosity functions to make predictions for the number density of quasars in upcoming surveys, predicting there should be ∼ 119 ± 28 (∼ 87 ± 28) quasars detectable in the F125W band of the WIDE (DEEP) fields of the Cosmic Assembly Near-infrared Deep Extragalactic Legacy Survey (CANDELS) from z = 5 − 6, ∼ 19 ± 7 (∼ 18 ± 9) from z = 6 − 7, and ∼ 1.7 ± 1.5 (∼ 1.5 ± 1.5) from z = 7 − 8. We also investigate quasar clustering, finding that the correlation length is fully consistent with current constraints for Sloan quasars (r 0 ∼ 17 h −1 Mpc at z = 4 for quasars above m i = 20.2), and grows slowly with redshift up to z = 6 (r 0 ∼ 22 h −1 Mpc). Finally, we note that the quasar clustering strength depends weakly on luminosity for low L BH , but gets stronger at higher L BH as the BHs are found in higher mass halos.
2008 3rd Petascale Data Storage Workshop, 2008
In this work we present an scientific application that has been given a Hadoop MapReduce implemen... more In this work we present an scientific application that has been given a Hadoop MapReduce implementation. We also discuss other scientific fields of supercomputing that could benefit from a MapReduce implementation. We recognize in this work that Hadoop has potential benefit for more applications than simply datamining, but that it is not a panacea for all data intensive applications. We provide an example of how the halo finding application, when applied to large astrophysics datasets, benefits from the model of the Hadoop architecture. The halo finding application uses a friends of friends algorithm to quickly cluster together large sets of particles to output files which a visualization software can interpret. The current implementation requires that large datasets be moved from storage to computation resources for every simulation of astronomy data. Our Hadoop implementation allows for an in-place halo finding application on the datasets, which removes the time consuming process of tranferring data between resources.
Monthly Notices of the Royal Astronomical Society, 2012
We examine predictions for the quasar luminosity functions (QLF) and quasar clustering at high re... more We examine predictions for the quasar luminosity functions (QLF) and quasar clustering at high redshift (z 4.75) using MassiveBlack, our new hydrodynamic cosmological simulation which includes a self-consistent model for black hole growth and feedback. We show that the model reproduces the Sloan QLF within observational constraints at z 5. We find that the high-z QLF is consistent with a redshiftindependent occupation distribution of BHs among dark matter halos (which we provide) such that the evolution of the QLF follows that of the halo mass function. The sole exception is the bright-end at z = 6 and 7, where BHs in high-mass halos tend to be unusually bright due to extended periods of Eddington growth caused by high density cold flows into the halo center. We further use these luminosity functions to make predictions for the number density of quasars in upcoming surveys, predicting there should be ∼ 119 ± 28 (∼ 87 ± 28) quasars detectable in the F125W band of the WIDE (DEEP) fields of the Cosmic Assembly Near-infrared Deep Extragalactic Legacy Survey (CANDELS) from z = 5 − 6, ∼ 19 ± 7 (∼ 18 ± 9) from z = 6 − 7, and ∼ 1.7 ± 1.5 (∼ 1.5 ± 1.5) from z = 7 − 8. We also investigate quasar clustering, finding that the correlation length is fully consistent with current constraints for Sloan quasars (r 0 ∼ 17 h −1 Mpc at z = 4 for quasars above m i = 20.2), and grows slowly with redshift up to z = 6 (r 0 ∼ 22 h −1 Mpc). Finally, we note that the quasar clustering strength depends weakly on luminosity for low L BH , but gets stronger at higher L BH as the BHs are found in higher mass halos.