Here are the basic concept of machine learning and some useful resources.


Background of Machine Learning:

Machine learning is a good approach for solving the dynamic optimization problems. A typical fully connected neural network formed by neurons consists of three layers: the input layer, the hidden layer and the output layer. Each link in the neural network has a weight and the weights of all links are randomly initialized, which will be updated according to the environment.

The deep Q-network and deep Q-learning (DQN) algorithm were developed and proposed by Google Deepmind. In DQN, a fixed neural network and a target neural network are used to learn the environment.

In the deep deterministic policy gradient (DDPG) algorithm, two pairs of neural networks are utilized: the actor neural network and the critic neural network, the target actor network and the target critic network.

Blog Image

For more details, please refer to the reference below:
L. Zhang, B. Jabbari and N. Ansari, “Deep Reinforcement Learning Driven UAV-assisted Edge Computing,” IEEE Internet of Things Journal, vol. 9, no. 24, pp. 25449-25459, Dec. 2022.


Deep Q Learning:

Here are some good DQN Python codes. They are all free to be downloaded. You need to use Python > 3.7 and Tensorflow >2.0 to run them.

Deep Reinforcement Learning:

Here are some good DQN Python codes. They are all free to be downloaded. You need to use Python > 3.7 and Tensorflow >2.0 to run them.

Userful Presentation Slides:

Machine Learning Publications:

Journals Articles:
  1. L. Zhang, B. Jabbari and N. Ansari, “Deep Reinforcement Learning Driven UAV-assisted Edge Computing,” IEEE Internet of Things Journal, vol. 9, no. 24, pp. 25449-25459, Dec. 2022.
Conference Papers:
  1. L. Zhang and B. Jabbari, “Machine Learning for Caching Placement in Edge Computing Networks,” accepted for publication in International Conference on Computing, Networking and Communications (ICNC), pp. 1-5, Feb. 2024 (Corresponding Author).
  2. L. Zhang and B. Jabbari, “Machine Learning Driven Latency Optimization for Application-aware Edge Computing-based IoTs,” in Proceedings of IEEE International Conference on Communications (ICC), pp. 183-188, May 2022 (Corresponding Author).
  3. L. Zhang, B. Jabbari and N. Ansari, “Machine Learning Driven UAV-assisted Edge Computing,” in Proceedings of IEEE Wireless Communications and Networking Conference (WCNC), pp. 2220-2225, Apr. 2022 (Corresponding Author).