In this work, we introduce FedML, an open research library and benchmark that facilitates the development of new 'federated learning algorithms' and fair performance comparisons. FedML supports three computing paradigms (distributed training, mobile on-device training, and standalone simulation) for users to conduct experiments in different system environments. We maintain the source code, documents, and user community at https://fedml.ai as well as at https://github.com/FedML-AI/FedML.