The discovery of potent therapeutic compounds against dengue virus is urgently needed. The NS2B-NS3 protease (NS2B-NS3pro) of dengue fever virus carries out all enzymatic activities needed for polyprotein processing and is considered to be amenable to antiviral inhibition by analogy. Virtual screening of 300,000 compounds using Autodock 3 on the GVSS platform was conducted to identify novel inhibitors against the NS2B-NS3pro. Thirty-six compounds were selected for in vitro assay against NS2B-NS3pro expressed in Pichia pastoris. Seven novel compounds were identified as inhibitors with IC50 values of 3.9 ± 0.6–86.7 ± 3.6 μM. Three strong NS2B-NS3pro inhibitors were further confirmed as competitive inhibitors with Ki values of 4.0 ± 0.4, 4.9 ± 0.3, and 3.4 ± 0.1 μM, respectively. Hydrophobic and hydrogen bond interactions between amino acid residues in the NS3pro active site with inhibition compounds were also identified.