Several semantic image search schemes have been recently proposed to retrieve images from the web. However, the query context is regularly ignored in these techniques and hence, many of the returned images are not adequately relevant. In this paper, we make use of context to further confine the outcome of the semantic search engines. For this purpose, we propose a hybrid search engine which utilizes concept and context for retrieving precise results. In the proposed model, an ontology is exploited for annotating images and accomplishing search process in the semantic level. Furthermore, the query of the user is modified with the concepts available in the ontology. Next, we make use of search context of the user and augment the query with the information extracted from the user’s context to additionally eliminate irrelevant results. Experimental results show that the combination of concept and context is effective in retrieving and presenting the most relevant results to the user.