use-case-and-architecture/ai_computing_force_scheduling/README.md

AI-based Computer Force Network Traffic Control and Computer Force Matching

Description:

Computer Force Network integrates distributed and ubiquitous computing capabilities in different geographic locations, and its sources include various computing devices such as cloud computing nodes, edge computing nodes, end devices, network devices, etc. The computing tasks in the CFN environment are large in volume and diverse in type, including data analysis, AI reasoning, graphics rendering, and other computing tasks. In this case, the traditional traffic control strategy may not be able to effectively handle the diversity and magnitude of tasks, which may lead to the waste of computing resources, delay of computing tasks, and degradation of service quality. To solve these problems, AI-based traffic control and computing force matching can be used to train AI models using deep learning algorithms by collecting a large amount of network traffic data, device state data, and task demand data. The model can not only learn the pattern of network traffic and computing tasks but also predict future traffic changes and task demands, as well as the computing capacity of devices, and adjust the traffic control strategy and arithmetic matching strategy in real-time based on this information.

With the help of AI, operators can manage traffic and computing power more effectively, reduce network congestion, improve the utilization of computing resources, reduce the latency of computing tasks, and improve the quality of service. For example, when a large number of data analysis tasks are predicted to be coming, AI systems can adjust network configurations in advance to prioritize allocating computing resources to these tasks to meet demand. When the capacity of computing devices is predicted to be insufficient to handle the upcoming tasks, the AI system can adjust the traffic control policy in advance to redirect some tasks to other devices to prevent congestion.

AI-based Computer Force Network traffic control and computer force matching bring significant performance improvements to large-scale CFN, enabling operators to manage computing resources better to meet the demands of various computing tasks.

Implementation:

Data collection: The dataset used is extracted and aggregated based on cluster-trace-v2018

Data pre-processing: Pre-process the collected data, including data cleaning, format conversion, feature extraction, etc.

Model selection training: According to the characteristics and needs of CFN, suitable AI models are selected for training. The training goal is for AI models to learn how to do workflow performance prediction.

Model testing and optimization: The trained AI models are tested in a simulated or natural environment, and the model is adjusted and optimized according to the test results.

Model deployment: The optimized AI model is deployed to CFN.

Real-time adjustment: The model needs to be dynamically adjusted and optimized according to the real-time network status and task demand data collected after deployment.

Model update: The model is regularly updated and optimized according to the network operation and model performance.

Continuous monitoring and adjustment: After the model is deployed, the network state and task execution need to be continuously monitored, the AI model needs to be adjusted as required, and the model needs to be periodically retrained to cope with changes in the network environment

Usage:

python run_CFN_schedule --model_name=path_of_model

Requirements:

• PyTorch & PyTorch Geometric
• Nvidia CUDA
• matplotlib
• numpy
• seaborn
• six
• tzdata
• zipp
• packaging
• pandas
• Pillow
• pyparsing
• python-dateutil
• pytz
• PyYAML
• contourpy
• cycler
• fonttools
• importlib-resources
• kiwisolver