LLM as a Judge` is an alternative to human evaluators for evaluating Large Language Models (LLMs) in language understanding tasks. It uses a powerful solution that employs LLMs to evaluate LLM responses based on specific criteria, providing a scalable and cost-effective way to assess model performance. The technique has gained popularity due to its ability to understand complex pieces of generated text and recognize nuances, making it an attractive alternative to traditional evaluation methods such as BERT and ROUGE. However, LLM judges have limitations, including narcissistic bias, more is more preference for verbose text, not-so-fine-grained evaluation scores, position bias, and hallucination issues. To address these limitations, techniques like chain-of-thought prompting, few-shot prompting, using probabilities of output tokens, reference-guided judging, confining LLM judgements, swapping positions, fine-tuning, and leveraging DeepEval's metrics can be employed to improve the accuracy and reliability of LLM judges. By incorporating these methods into an LLM evaluation metric, it is possible to create a comprehensive suite of evaluation results that can be used to benchmark, evaluate, and even regression test LLM systems.