Trading redundancy for communication: Speeding up Distributed SGD for Non-convex Optimization \

Keyphrases

• Redundancy 100%
• Nonconvex Optimization 100%
• Model Averaging 50%
• Order of Accuracy 25%
• Training Data 25%
• Gradient Vector 25%
• Communication Efficiency 25%
• Communication Overhead 25%
• Communication Cost 25%
• Updating Model 25%
• Distributed Environment 25%
• Beneficial Aspects 25%
• Periodic Models 25%
• Stochastic Algorithm 25%
• Residual Error 25%
• Distributed Optimization Algorithm 25%
• Communication Round 25%
• Wide Neural Network 25%
• CIFAR10 Dataset 25%
• ImageNet Dataset 25%
• Local Update 25%
• CIFAR-100 25%

Computer Science

• Convex Optimization 100%
• Neural Network 50%
• Gradient Vector 50%
• Training Data 50%
• Communication Overhead 50%
• Communication Cost 50%
• Distributed Environment 50%
• Distributed Optimization (Algorithms) 50%
• Stochastic Algorithm 50%
• Residual Error 50%