Our work brings to light that Content-Based Image Retrieval (CBIR) systems, which are commonly used in image search engines, can be potential attack targets of adversaries. In this work, we present the threat model of evading the CBIRs. Specifically, we focus our work on the SIFT/SURF based CBIRs and propose several algorithms for removing/injecting the key points in images to bypass the algorithms. We apply the RMD algorithm and our algorithm to remove the SIFT and SURF key points respectively. Moreover, we inject SIFT key points into images with our IMD algorithm (inverting the operation of RMD) or surround an image with a frame filled with ‘basic bricks'. We evaluate the algorithms on an image indexing engine VisualIndex with three strategies: removal only, injection only and hybrid. The experimental results show the effectiveness of bypassing the engine. With the algorithms and strategies, we succeed in evading Google Image Search Engine, which can be considered as a black-box CBIR system, while the utility of the image is preserved. We also demo the possibility of source/target attack. To conclude, our work proves the existence of threats to CBIR systems and demonstrates that industrial-level Image Search Engines, such as Google Image Search, are prone to be attacked with adversarial images.