The proliferation of IEEE 802.11 or WiFi networks, and the explosive growth in traffic demands call for solutions to maximize the capacity of WiFi networks. Hence, maximizing the spatial reuse of WiFi networks is critical as doing so allows multiple concurrent transmissions. In this respect, a critical network parameter, Clear Channel Assessment (CCA) threshold, plays a vital role as it dictates whether a node is allowed to transmit after sensing the channel. In this paper, we propose to use Deep Q-network (DQN) under two learning patterns to select the CCA threshold of devices. We further consider Transmit Power Control (TPC) in conjunction with CCA threshold selection to improve the capacity of a WiFi network. The simulation results show that our approach is capable of selecting the optimal CCA threshold for each device. As a result, the average throughput is 62.4% higher than that of a legacy Dynamic Sensitivity Control (DSC) algorithm.