How do you evaluate and compare different quantum software frameworks and tools?

When it comes to quantum computing, unlike traditional computing, we have limited options to choose from. Therefore, there is a high probability that people would opt for widely used quantum software development kits (SDKs) such as QisKit and Pennylane. Because we need a tool that allows us to create any algorithm we desire, simplifies the implementation of quantum circuits, and is compatible with the majority of existing QPUs. Additionally, it should have a strong and supportive community, well-documented and reliable functionality, and enough implementation to instill confidence in the tool's credibility.

The approach for using quantum SDK is very different than the traditional computing. In traditional computing a lot of high level languages with libraries to do specific tasks are available, while in quantum computing it is still at QPU instruction set level. Compare it with MSAM or TSAM assembly languages that was used in early '80s, difference is in low level machine instructions traditional computing broke everything down to algebraic problems, however with QPU it is possible to approach a problem at a statistical and probabilistic way. And Going forward I am hopeful that Quantum SKs will support non- mathematical approach of solving problems.

1) Design performance metrics: quality of solution, time to compute, cost. 2) Identify approaches that you would like to benchmark against the metrics. 3) Make sure that you have adequate computational resources in order to evaluate the problem for a variety of approaches.

Consider compatibility with various quantum hardware and simulators, the ease of use and programming language, and the quality of community support and documentation. Key assessment features include the range of functionalities, performance, scalability, and integration capabilities with classical software systems. It's essential to distinguish between open-source and proprietary options, assess the frequency of updates and developer support, and consider security aspects for sensitive data. Finally, real-world applications and case studies can provide valuable insights into the practical effectiveness of each framework.

It is imperative to work with good, solid, real world use cases. If you are just starting out then pick one that has been tried and tested over and gain. Create your own solution and benchmark against existing solutions. In my experience many Quantum Curious people will get stuck, by not having a well defined use case, in the experimentation stage which may lead to disillusionment, frustration and ultimately failure. Find and stick to a well defined use case and work with communities to support.

One key consideration in your evaluation should be availability and connectivity to non-quantum data science and machine learning capabilities. In the course of developing, testing, and training your quantum solution, you are going to be generating significant quantities of data. Contained in that data will be significant clues and markers to how you might be able to optimize, debug, and reduce noise; there is also significant possibility for entirely new discoveries. Access to tools like graph databases and sophisticated unsupervised machine learning capabilities can significantly improve the quality and productivity of your efforts.

Compatibility with different quantum hardware and platforms is a significant consideration. Some software frameworks are optimized for certain quantum processors or specific architectures, like superconducting qubits or trapped ions, which affects their performance and applicability.

It does amaze me how often people seem to omit Quantum Annealing from this list? When pointing people to Quantum Hardware and Platforms, Quantum Annealing is available to solve real world solutions right now, today. Quantum Annealing is particularly suited to Optimization problems. All the tools and access to the Quantum Computing platform is available today. It depends on what your aim is. If its Quantum Logic Gate Research then thats one route; possible 5-7 years away from actual real world value. The alternative is start with Quantum Annealing and solve problems that matter today nd gain valuable skills along the way.

Assess compatibility with available quantum hardware platforms (e.g., IBM Quantum, Google Quantum, Rigetti). Consider support for different quantum computing models (e.g., gate-based, adiabatic, topological). Evaluate the level of abstraction provided by the framework in interacting with quantum hardware.

Some frameworks are designed to work with specific quantum processors, offering optimized performance and access to proprietary tools (like Rigetti's Forest for its own quantum processors). Q# and the Quantum Development Kit, aim for a more hardware-agnostic approach, focusing on simulation capabilities and the ability to target multiple quantum computing platforms. Assessing the availability of quantum hardware, either through cloud access or physical devices, and the framework's ability to efficiently leverage these resources is essential for practical quantum computing projects.

It is important to select not only for functional capability but access to skilled resources. A language with differentiated capabilities is of limited value if nobody knows how to use it, or if the necessary educational resources are hard to come by or prohibitively costly. When choosing not just the programming language but other elements of the development stack, make a shortlist for each component and investigate how hard it will be to hire or train a team to use it.

The survey run by Unitary Fund has shown that Qiskit has lost a significant proportion of users when comparing 2022 to 2023. Tket by Quantinuum, on the other hand, while still with only a small fraction of the total users seems to be on the rise. Always seek for new frameworks that fit your goals. Every task is different and the same SDK might not fit all the needs.

Languages like Q# and Qiskit have a high-level programming interface for quantum computing. They incorporate elements from classical programming while introducing concepts like qubits and quantum gates. These languages support a variety of quantum algorithms and enable developers to experiment with quantum computing models, including circuit-based and measurement-based quantum computing. They facilitate the exploration of quantum computing's potential and the development of applications across cryptography, optimization, and simulation.

The choice of quantum programming languages and paradigms is essential. Some frameworks use languages that are extensions of classical programming languages (like Qiskit with Python), while others use entirely new quantum-focused languages. (Q#, Qutip)

Quantum paradigms are mainly a conceptual models for expressing quantum information, like circuit-based or measurement-based. Choosing a framework depends on preferred languages and paradigms, e.g. Qutip for Python, PennyLane for measurement-based. Consider the learning curve, expressiveness, flexibility, and portability of the languages and paradigms. The right combination of language, paradigm, and framework is crucial for efficient quantum programming and aligning with project requirements. The languages which we can use are Python , C++, Q#, Quipper, F#, Rust, Julia and many more. It depend on the library which you want to use.

To evaluate quantum software frameworks, examine their libraries and packages. These collections of functions and algorithms, like QAOA, VQE, or QFT, are crucial for development. For instance, frameworks like Qiskit Aqua, Cirq OpenFermion, or Q# Quantum Chemistry offer different integrations. At White Internet Consulting, we compared these tools based on completeness, reliability, documentation, and community support. In one project, Qiskit Aqua’s comprehensive documentation and active community proved invaluable. My advice: choose frameworks that best align with your project's needs and ensure they are well-supported and documented to facilitate development.

Some quantum software libraries and packages I found useful (this is a brief list as they are many more): Qiskit: open-source allows to work with IBM's QC and simulators. Used for creating, simulating, and running quantum algorithms. Cirq: is a Python library for writing, manipulating, and optimizing quantum circuits and running them against quantum computers and simulators. ProjectQ: open-source software framework for QC, allows to implement quantum programs in Python and run them on either simulators or actual quantum hardware. Forest: include PyQuil, the Quil language for writing quantum algorithms, and a QVM Strawberry Fields: for simulating and optimizing quantum optics circuits, focused on photonic QC.

Examining quantum software libraries and packages is a pivotal step in selecting the right framework for quantum development. These libraries, like Qiskit Aqua or Cirq OpenFermion, house crucial functionalities such as QAOA or VQE algorithms. For instance, Qiskit Aqua's extensive library not only simplifies implementation but also offers robust documentation and a thriving community for support. Ensuring compatibility and reliability of these libraries with your chosen framework is essential for seamless integration and efficient development in the rapidly evolving quantum computing landscape.

I am very much into Qiskit, I believe it is at the level that rethinking many fundamental computation problems are possible, but it also kind of assumes the computation is a mathematical problem. The other options are Cirq: for building, modifying, and running quantum algorithms on Noisy Intermediate-Scale Quantum (NISQ) devices. PyQuil: to write quantum programs in Python and execute them on Rigetti's quantum computers.ProjectQ: to write quantum algorithms in Python and run them on a simulator or a real quantum computer. Microsoft QDK: Q#, simulator for running quantum algorithms and interfaces for running them on Microsoft's quantum computers. QuTiP, Strawberry Fields: library for developing photonic quantum computing algorithms.

"You might also need to consider the completeness, the reliability, the documentation, and the community of the quantum software libraries and packages." This is a crucial point. Many are incomplete, badly documented or have a not great community. I can't say much about the reliability, but the quantum community as a whole needs to step up in these areas.

Be sure to include a data visualization tool in your development environment. Understanding quantum computing is hard, and requires specialized training; at various points in your project you are going to have to explain features, results, and delivery issues to stakeholders who don't have that training. Good visualizations can be incredibly powerful ways to get your message across.

The probabilistic nature of quantum measurements and the complexity of entangled states are some challenges when testing and debugging. Frameworks that offer comprehensive testing suites, simulation tools, and debugging aids ease the development process. Tools that allow step-by-step simulation of quantum circuits or visualization of quantum states can help identify and correct errors more efficiently.

This is a very rare feature in quantum. The best example, off the top of my head, is Fire Opal, which validates circuits prior to execution. If you try to submit an 8-qubit circuit to a 7-qubit backend, for example, it will tell you before you wait in a queue for an error.

Error Sleuths: Quantum computing is inherently noisy, so robust testing and debugging tools are crucial. Frameworks might offer features like unit testing, simulation, and error correction capabilities. Accuracy Matters: Evaluate the effectiveness and speed of these tools. Remember, debugging can be an iterative process, so user-friendliness and informative feedback are key. Example: While debugging your quantum circuit on Google's Quantum AI platform, you'd utilize Cirq's built-in simulators and tools like the Quantum Virtual Machine to identify and fix errors before running it on real hardware, which can be expensive and time-consuming.

Yes, there is debugging, and there is testing. A platform can debug, up to a point, but for now, most quantum code isn't as complicated as software libraries. Getting it to work however, that is a long trial and error based thing. There are few options there. Maybe improving it means changing your characterization of your Transmons, maybe it means not using some frequencies in your colour centers, or maybe it is as simple as just not using some qubits for certain lines. Either way, you run, you benchmark, you improve. And continue to do so. No QC is the same, and they often evolve over time a bit. Knowing this will help you produce value. But, ideally, a supplier or quantum systems integrator does that part for you.

Consider dependencies. A suite of libraries that peacefully coexist today can magically stop working tomorrow due to updates. So, before updating anything, always assume something will break and prepare accordingly.

What is missing in this article is objective benchmarks that actually evaluate the performance of specific frameworks and approaches e.g. in transpiling circuits for a given hardware. For the given example, this can be done by counting the number of 1- and 2-qubit gates a transpiled circuit boasts with benchmarks like MQT bench from TU Munich.

In my experience, all game changing technologies miss one thing, meeting the business user in the applications where they spend their time. Its all well and good creating a Quantum Solution, if its not deployed in a simple and consumable manner then it may fail. Business People spend their time in Spreadsheets, Visualisation tools, Dashboards, Sales & Marketing Platforms etc For example not many people understand or even care how a CPU works they just care they can send an email. Its the same with Annealing Quantum Computing, its a powerful computing architecture that delivers valuable optimisation results faster and better directly to your familiar business applications.

The ease of deploying and maintaining quantum software is also a critical factor. This includes the integration of quantum solutions with existing IT infrastructure, the scalability of the software, and the support provided by the developers or community for updates and troubleshooting.

Deployment Dynamos: Frameworks often provide tools for version control, configuration management, and even cloud deployment options. Choose one that aligns with your deployment needs and preferences. Future-Proofing: Consider the scalability, security, and ongoing support offered by the framework. You'll want a solution that can grow with your project and ensure long-term maintainability. Example: You've built a complex quantum machine learning model and want to deploy it for cloud-based inference. Tools like Qiskit Terra or Microsoft's Q# Azure offer deployment options and integrations with cloud platforms, allowing you to manage and scale your model effectively.