How do you decide when to turn off a feature flag in a mobile app?

In my experience, defining success criteria is crucial for any feature rollout. Recently at work, we launched a few examples for users to implement our product without clear success/usage metrics. It led to confusion about the impact on user engagement and the success of the work. From this, I learned the importance of setting specific, measurable goals before launching. For future rollouts, I make sure to define key performance indicators, which will significantly improve our decision-making process.

- Clear goals for the feature. - Defects and issues resulting in degraded customer experience - Customer feedback Any of the above should be a trigger to pulling plug on the feature. Flag is one of many ways to make it happen.

Success Criteria is a must when you run or will run the project. It will give you a purpose and setting the expectation to our business partner or users. Success criteria is can be higher level like KPI or even go down to lower level to requirement level. Again, i will say defining success criteria is a must

The primary decision on whether to add a feature is also dependent on whether there is a need, a user experience benefit or a problem to solve. Measuring user adoption is also key after implementing to gauge success and usability

In my experience, it's crucial to always pair the success criteria with guardrail metrics. Sometimes it's easy to overlook second order effects. Those effects can take time to become material and might cause you to regret the earlier decision. One time, we were able to increase new user onboarding rate by eliminating a friction in the process. By means of A/B testing, this was a success at the time. Months later, we noticed a decline in the overall system quality. It turns out, that friction was forcing new users to invest more into the product and hence raised the overall quality.

Monitoring and analyzing data is essential for gaining insights into performance and making informed decisions. Implement robust data monitoring tools to track key metrics and capture relevant information. Regularly analyze the collected data to identify patterns, trends, and anomalies, allowing you to understand user behavior and system performance. Use data-driven insights to refine strategies, optimize processes, and enhance overall performance, fostering continuous improvement and informed decision-making.

Success criteria has 2 parts functional features and non functional features (NFRs). There is a big tendency to look at functional features meeting the exit criteria with a big fight going on between Dev and QA team. The elephant in the room is NFR which either is overlooked or taken up in the end leaving little room for stability and regression. It is critical to address both of these simultaneously, Product Manager and Technical Architect play this role and need to be tightly connected on the success factor

D'après mon expérience, les solutions de feature flags sont à peine utiles si on se limite strictement à cette fonctionnalité. Il est important de récolter de la donnée dans le contexte du flag, et ainsi de mesurer ce qui se passe bien ou pas pour les utilisateurs qui ont le flag activé. Bien s?r, les données récoltées ne doivent vraiment pas se limiter aux données techniques (erreur, bugs ou autres) mais également contenir des données orientées ? produit ? (comme le comportement utilisateur par exemple). Pour avoir des données utilisables pour prendre des décisions, je pense qu'il est nécessaire de déployer une feature de fa?on très progressive (en fonction de la taille de la base utilisateur). Ceci permet la comparaison.

I believe that the decision should be mostly data driven but compared with success criteria. Tools like Firebase Analytics combined with BigQuery can provide a lot of decision taking information. Users can also be asked to fill simple surveys which add more data to analyze and add to a decision. Another angle to thIs could be that the new feature is not working well and needs to be temporarily turned off. Tools like Firebase Crashlytics are essential and provide detailed information not just for specific features but a whole application.

Once your feature flag is up, data becomes your best friend. It’s like gathering puzzle pieces – user feedback, app performance stats, engagement levels. Tools like Firebase or Amplitude can be your allies here. This phase is about observing trends, understanding user behavior, and spotting any unexpected outcomes. The key is to keep an eye on the data that directly ties back to your success criteria. It’s not just about collecting data; it’s about finding the story it tells.

La clave es pensar en el objetivo final de la aplicación móvil, y para ello se debe definir un camino en donde se vayan alcanzado metas parciales, las que darán pie pa generar cambios, por ejemplo a medida que aumenta el número de usuarios, la retención, el tiempo de uso, el impacto de una función específica, que se cumpla una fecha o hito de negocio, etc. En mi experiencia no hay una fórmula única ya que cada aplicación/negocio es diferente, pero en cualquier caso la decisión pasará por el cumplimiento de las metas específicas, y por la correcta medición de los datos asociados, posteriormente se deberá evaluar el impacto del cambio para decidir si se continúa o si se requieren ajustes, por lo que es un proceso continuo.

Personally, I find the approach of managing feature flags based on data and success criteria to be a smart and efficient way to develop and refine software. It's like a compass that helps guide decisions, ensuring that the features we introduce are well-received and add value to users. Celebrating successes and learning from failures is not just a professional practice but also a rewarding journey of growth and innovation. It's all about staying agile and committed to delivering the best possible user experience, which is something I find quite satisfying in the field of software development.

When making decisions, leverage your data and align them with your predefined goals. Analyze key performance indicators and relevant metrics to ensure decisions are grounded in factual insights rather than assumptions. Regularly revisit your goals to ensure they remain relevant and adjust your strategy based on the evolving data landscape. This data-driven decision-making approach fosters adaptability, precision, and a more effective pursuit of organizational objectives.

Feature flag is not a fixed set of features, in our experience the defined features would change of a feature flag over 4 sprints based on staged releases. But overall goal mostly remains the same, subtasks change.

We can also pilot a feature using a flag in a particular demographic and evaluate the impact it brings on to the product and implement across the based on the feedback and product vision

When turning off a feature it is vital to also make a plan, inform, collaborate and communicate to all teams involve in the rollout as well as the actual users. You can be surprise of the feedback you can receive and learn from it.

Deciding when to turn off a feature flag in a mobile app involves effective communication and collaboration within the team. Regularly assess user feedback, monitor performance metrics, and engage in team discussions to evaluate the feature's impact. Prioritize input from various stakeholders, including developers, QA, and product managers. Establish clear criteria for success and potential risks. When consensus is reached, communicate the decision transparently to the team, ensuring everyone is informed and aligned. This collaborative approach ensures informed and timely decisions regarding feature flag management.

Effective communication and collaboration with your team are fundamental for success. Foster an open and transparent communication culture, ensuring that information flows seamlessly among team members. Establish clear channels for collaboration, encouraging the sharing of ideas and feedback. Regular team meetings and updates enhance coordination, aligning everyone towards common goals. Cultivate a supportive environment that values collaboration, empowering each team member to contribute their unique skills and insights.

Absolutely. Communication and collaboration are key components. Involving your team and stakeholders in the decision-making process ensures alignment and shared understanding. Documenting findings and sharing learnings fosters a culture of transparency, enabling the team to collectively improve and innovate in future development cycles.

This is one of the key thing as the people on ground will have many insights which a typical architect/product owner/manager won't. Build a learning mindset in the team and they will guide you in right direction.

Similar to Software Development Lifecycle make sure to define a lifecycle for your feature flags. Answer the question in which phase the feature flag is used + Release + PoC + Ops Feature flag lifecycle for PoC might look differently than for Ops related to longevity (week vs year) for example.

Regularly reviewing and optimizing your feature flagging process is crucial for maximizing its effectiveness. Evaluate the success of feature flags in achieving project goals and gather feedback from development and testing teams. Identify any bottlenecks or challenges in the feature flag lifecycle and streamline the process for efficiency. Continuous improvement ensures that your feature flagging strategy remains agile and aligns with evolving project requirements, fostering a more responsive and adaptable development environment.

Conduct a post-implementation review of the feature flagging process itself. Evaluate how effectively the feature flagging system facilitated testing, deployment, and monitoring. Continuously optimize your feature flagging process based on lessons learned from each feature release. This includes refining success criteria, improving monitoring capabilities, and enhancing collaboration among team members.

To optimize your feature flagging process, periodically evaluate its effectiveness and efficiency. Focus on areas like automating workflows to reduce manual effort, adopting a feature flag management platform for better control, and establishing a clear feature flagging policy for consistent practices across teams. These improvements ensure your strategy remains agile, accurate, and in line with your development goals, enhancing overall project efficiency and team collaboration.

Flagging process must be automated. Lack of automation leads to a pack of old and unused code and this can produce a lot of bugs in future.

Feature flags, whether used in our local development or remotely, give us the flexibility and confidence we need to work more efficiently. They allow us to take iterative steps in building and improving things, promoting a creative approach. Having these toggles in place during development and after we release a feature means our teams can make big strides without getting stuck on trying to make everything absolutely perfect. It's all about making progress rather than chasing after perfection.

We would turnoff/disable the feature flag as soon as we notice a critical bug/issue in that particular feature. Issue/crash for the feature can be fixed later , as users are no longer affected.

Before deciding on when to turn off a feature flag, it is important to understand and consider what is a feature flag. A Feature Flag is the ability in an application to turn a feature on and off. Feature Flags are often used by teams to control access to a new feature and for feature experimentation. Feature Flags are also used to reduce the risk of deploying software by disconnecting the deployment of the code from the enablement of the new feature.

In case the feature rollout does not go as planned, have a rollback plan in place so that you can quickly revert to the previous version of the app.

Consider integrating user feedback, conducting real-world scenario testing, fostering cross-functional collaboration, prioritizing security and compliance, addressing user privacy concerns, implementing robust version control, maintaining thorough documentation, assessing performance impact, and educating users to enhance the comprehensiveness and effectiveness of your feature flagging approach.