The Secret Sauce of Tough Problem Solving on Computers/HPC/Clusters
Everyone asks me the secret to our success. Here are some pointers.
(1) Never use a functional language. Always use a plan old OOPs Language.
(2) Don't go for Fast Execution/C++ initially. Go for easy bug free programming in Java/Python to develop Fastest Algorithms/Solutions.
(3) Never use recursion. After 20-30 depth they konk.
(4) Watch out for memory bottlenecks. N^2 memory for 10^6 size is 10^12 Oooops!
(5) To solve really hard problems you have to run systems 24x7. Prevent CPU Over heating and abrupt cluster shutdowns
(6) Avoid GPGPU's, always use CPU's/Xeon Phi and do Dynamic Programming
(7) Always try to come up with Deterministic Algorithms, after/failing which you try Fast Approximation Algorithms.
(8) Maximize Reduction of Complexity/Dimensions/Search-Space by including all constraints you can into the solution.
(9) Always attack the toughest version of the problem. Once solved you can easily reduce/adapt it to other forms and solve dozens of easier problems.
(10) R'ber to celebrate little successes. Nobody around you will understand a crap about what you are trying to achieve or appreciate your efforts. Over years and decades you will eventually celebrate huge successes.