Cross-drainage hydraulic structures such as culverts and bridges in urban landscapes are prone to get blocked by the transported debris (e.g., urban, vegetated), which often reduces their hydraulic capacity and triggers flash floods. Unavailability of relevant data from blockage-originated flooding events and complex nature of debris accumulation are highlighted factors hindering the research within the blockage management domain. Wollongong City Council (WCC) blockage conduit policy is the leading formal guidelines to incorporate blockage into design guidelines; however, are criticized by the hydraulic engineers for its dependence on the post-flood visual inspections (i.e., visual blockage) instead of peak floods hydraulic investigations (i.e., hydraulic blockage). Apparently, no quantifiable relationship is reported between the visual blockage and hydraulic blockage; therefore, many consider WCC blockage guidelines invalid. This paper exploits the power of Artificial Intelligence (AI), motivated by its recent success, and attempts to relate visual blockage with hydraulic blockage by proposing a deep learning pipeline to predict hydraulic blockage from an image of the culvert. Two experiments are performed where the conventional pipeline and end-to-end learning approaches are implemented and compared in the context of predicting hydraulic blockage from a single image. In experiment one, the conventional deep learning pipeline approach (i.e., feature extraction using CNN and regression using ANN) is adopted. In contrast, in experiment two, end-to-end deep learning models (i.e., E2E_ MobileNet, E2E_ BlockageNet) are trained and compared with the conventional pipeline approach. Dataset (i.e., Hydraulics-Lab Blockage Dataset (HBD), Visual Hydraulics-Lab Dataset (VHD)) used in this research were collected from laboratory experiments performed using scaled physical models of culverts. E2E_ BlockageNet model was reported best in predicting hydraulic blockage with R2 score of 0.91 and indicated that hydraulic blockage could be interrelated with the visual features at the culvert.