Exploring Efficient Methods for Transformer-based Foundation Language Models