There are two major challenges to solving constrained optimization problems using a Quadratic Unconstrained Binary Optimization or QUBO solver (QS). First, we need to tune both the underlying problem parameters and the algorithm parameters. Second, the solution returned from a QS might not be feasible. While it is common to use automated tuners such as SMAC and Hyperopt to tune the algorithm parameters, the initial search ranges input for the auto tuner affect the performance of the QS. In this paper, we propose a framework that resembles the Algorithm Selection (AS) framework to tune algorithm parameters for an annealing-based QS. To cope with constraints, we focus on permutation-based combinatorial optimization problems, since computing the projection to the feasible space for this class of problems can be done efficiently; and for simplicity, the number of problem parameters can be reduced to one and we fix it. Methodologically, we train a recommendation system to to learn good annealing problem parameter ranges. During testing, we search for good hyperparameter values using a recommendation system approach. To illustrate our approach experimentally, we use the Fujitsu Digital Annealer as our QUBO solver and Optuna as the auto tuner to solve the Traveling Salesman Problem.