Recent advancements in large machine learning models have demonstrated that increasing the number of parameters enhances computational precision and unlocks capabilities once deemed unattainable. This trend is exemplified by the rapid growth in model sizes, for instance, GPT-3 contained 175 billion parameters, while GPT-4 reportedly utilizes up to 1.8 trillion. This trajectory is expected to continue in the foreseeable future. However, the explosive growth in model size presents two major challenges for computer architecture and systems research: prolonged simulation times, which can extend from several days to weeks for large-scale models, and infeasibility of deploying workloads on a single compute engine (e.g., a graphics processing unit (GPU)) due to limited on-device memory capacity. To address these challenges, this project proposes the development of scalable simulation techniques and advanced memory management strategies tailored for large-scale machine learning workloads on GPUs. Unlike existing application-agnostic approaches, this research will leverage the distinctive data access patterns and value distributions of modern machine learning models to enable more efficient memory compression and more accurate simulation acceleration. While the primary focus will be on emerging machine learning models, the broader objective is to advance GPU computing to better accommodate any big data workload constrained by memory limitations. This will facilitate faster and broader