Fog computing, as an extension of the cloud, provides computing resources for Internet of Things (IoT) applications through communicative fog nodes located at the network edge. Fog nodes assist cloud services in handling real-time applications by bringing the processing capability to where the data is generated. However, the introduction of fog nodes increases scheduling openness and uncertainty. The scheduling issues in fog computing need to consider the geography, load balancing, and network latency between IoT devices, fog nodes, as well as the parent cloud. Besides, the scheduling methods also need to deal with the occurrence of uncertain events timely so as to ensure service reliability. This paper proposes an agent-based framework with a decentralized structure to construct the architecture of fog computing, with considerations of the scheduling, load balance, and rescheduling processes.