Goal:
Track Insights Operator Data Enhancements epic in 2024

Provide a simple way to get a VM-friendly networking setup, without having to configure the underlying physical network.

Placeholder feature for ccx-ocp-core maintenance tasks.

tldr: three basic claims, the rest is explanation and one example

1. We cannot improve long term maintainability solely by fixing bugs.
2. Teams should be asked to produce designs for improving maintainability/debugability.
3. Specific maintenance items (or investigation of maintenance items), should be placed into planning as peer to PM requests and explicitly prioritized against them.

While bugs are an important metric, fixing bugs is different than investing in maintainability and debugability. Investing in fixing bugs will help alleviate immediate problems, but doesn't improve the ability to address future problems. You (may) get a code base with fewer bugs, but when you add a new feature, it will still be hard to debug problems and interactions. This pushes a code base towards stagnation where it gets harder and harder to add features.

One alternative is to ask teams to produce ideas for how they would improve future maintainability and debugability instead of focusing on immediate bugs. This would produce designs that make problem determination, bug resolution, and future feature additions faster over time.

I have a concrete example of one such outcome of focusing on bugs vs quality. We have resolved many bugs about communication failures with ingress by finding problems with point-to-point network communication. We have fixed the individual bugs, but have not improved the code for future debugging. In so doing, we chase many hard to diagnose problem across the stack. The alternative is to create a point-to-point network connectivity capability. this would immediately improve bug resolution and stability (detection) for kuryr, ovs, legacy sdn, network-edge, kube-apiserver, openshift-apiserver, authentication, and console. Bug fixing does not produce the same impact.

We need more investment in our future selves. Saying, "teams should reserve this" doesn't seem to be universally effective. Perhaps an approach that directly asks for designs and impacts and then follows up by placing the items directly in planning and prioritizing against PM feature requests would give teams the confidence to invest in these areas and give broad exposure to systemic problems.

Relevant links:

• Documentation:
  • Edge Diagnostics Scratchpad, our team's internal diagnostic guide.
  • Troubleshooting OCP networking issues - The complete guide, the SDN team's diagnostic guide.
  • Linux Performance, Brendan Gregg's guide to analyzing Linux performance issues.
  • RFC: A proper feedback loop on Alerts.
  • OpenShift Router Reload Technical Overview on Access.
  • Performance Scaling HAProxy with OpenShift on Access.
  • How to collect worker metrics to troubleshoot CPU load, memory pressure and interrupt issues and networking on worker nodes in OCP 4 on Access.
  • OpenShift Performance and Scale Knowledge Base on Mojo, results from OpenShift scalability testing.
  • Scalability and performance, OCP 4.5 documentation about the router's currently known scalability limits.
  • Scaling OpenShift Container Platform HAProxy Router, OCP 3.11 documentation about the manual performance configuration that was possible in OCP 3.
  • Timing web requests with cURL and Chrome from the Cloudflare blog.
  • tcpdump advanced filters, some useful tcpdump commands.
  • OpenShift SDN - Networking, OCP 3.11 documentation on the SDN (useful background reading).
  • Ingress Operator and Controller Status Conditions, design document for improved status condition reporting.
  • Observability tips for HAProxy, a slide deck by Willy Tarreau.
  • Interesting Traces - Out of Order versus Retransmissions, analysis using tshark.
  • The PCP Book: A Complete Documentation of Performance Co-Pilot, by Yogesh Babar.
  • Debugging kernel networking bug, brief guide to using SystemTap on RHCOS.
  • Troubleshooting throughput issues from the OCP 4.5 documentation.
  • Troubleshooting OpenShift Clusters and Workloads.
  • Red Hat Enterprise Linux Network Performance Tuning Guide (PDF).
  • openshift/enhancements#289 stability: point to point network check, a diagnostic built into the kube-apiserver operator.
• Diagnostic tools:
  • dropwatch to watch for packet drops.
  • ethtool to check NIC configuration.
  • iovisor/bcc: BCC - Tools for BPF-based Linux IO analysis, networking, monitoring, and more to trace and diagnose various issues in the networking stack.
  • r-curler to gather timing information about HTTP/HTTPS connections.
  • route-monitor, to monitor routes for reachability.
  • hping(3), a programmable packet generator.
  • OpenTracing / Jaeger in OpenShift.
  • node-problem-detector, a possible integration point for new diagnostics.
  • Using SystemTap by Brendan Gregg.
  • DTrace SystemTap cheatsheet (PDF).
• Visualization and more sophisticated diagnostic tools:
  • eldadru/ksniff, kubectl plugin for tcpdump & Wireshark.
  • ironcladlou/ditm, Dan's "Dan in the Middle" tool.
  • Skydive, network diagnostic and visualization tool.
  • ali, a "load testing tool capable of performing real-time analysis" with visualization.
• Testing tools:
  • stress-ng, a general stress-loading tool (CPU, filesystem, network, ...).
  • mb, the networking benchmarking tool written and used by Jiri Mencak from our Perf+Scale team.
• Case studies:
  • BZ1763206 is an example of diagnosing DNS latency/timeouts.
  • BZ1829779 Investigation details the diagnosis of route latency.
  • BZ1845545 is an example of diagnosing misconfigured DNS for an external LB.
  • Debugging network stalls on Kubernetes, from the GitHub Blog, about diagnosing Kubernetes performance issues related to ksoftirqd.

Feature Overview

As a cluster-admin, I want to run update in discrete steps. Update control plane and worker nodes independently.
I also want to back-up and restore incase of a problematic upgrade.

Background:

This Feature is a continuation of https://issues.redhat.com/browse/OCPSTRAT-180.
Customers are asking for improvements to the upgrade experience (both over-the-air and disconnected). This is a feature tracking epics required to get that work done.Below is the list of done tasks.  

1. OTA-700 Reduce False Positives (such as Degraded) 
2. OTA-922 - Better able to show the progress made in each discrete step 
3. [Cover by status command]Better visibility into any errors during the upgrades and documentation of what they error means and how to recover. 

Goals

1. Have an option to do upgrades in more discrete steps under admin control. Specifically, these steps are: 
   • Control plane upgrade
   • Worker nodes upgrade
   • Workload enabling upgrade (i..e. Router, other components) or infra nodes
2. An user experience around an end-2-end back-up and restore after a failed upgrade 
3. MCO-530 - Support in Telemetry for the discrete steps of upgrades 

References

• OpenShift Upgrades Customer Issues Discussion -6/22/2023
• Select Customer Experiences and Feedback on Upgrades

Feature Overview

This feature aims to enable customers of OCP to integrate 3rd party KMS solutions for encrypting etcd values at rest in accordance with:

https://kubernetes.io/docs/tasks/administer-cluster/kms-provider/

Goals

• Bring KMS v2 API to beta|stable level
• Create/expose mechanisms for customers to plug in containers/operators which can serve the API server's needs (can it be an operator, something provided via CoreOS layering, vanilla container spec provided to API server operator?)
• Provide similar UX experience for all of self-hosted, hypershift, SNO scenarios
• Provide example container/operator for the mechanism

General Prioritization for the Feature

1. Approved design for detection & actuation for stand-alone OCP clusters.
   1. How to detect a problem like an expired/lost key and no contact with the KMS provider?
   2. How to inform/notify the situation, even at node level, of the situation
2. Tech Preview (Feature gated) enabling Kube-KMS v2 for partners to start working on KMS plugin provider integrations:
   1. Cloud: (priority Azure > AWS > Google)
      1. Azure KMS
      2. Azure Dedicated HSM
      3. AWS KMS
      4. AWS CloudHSM
      5. Google Cloud HSM
   2. On-premise:
      1. HashiCorp Vault
      2. EU FSI & EU Telco KMS/HSM top-2 providers
3. GA after at least one stable KMS plugin provider

Feature Overview (aka. Goal Summary)

Volume Group Snapshots is a key new Kubernetes storage feature that allows multiple PVs to be grouped together and snapshotted at the same time. This enables customers to takes consistent snapshots of applications that span across multiple PVs.

This is also a key requirement for backup and DR solutions.

https://kubernetes.io/blog/2023/05/08/kubernetes-1-27-volume-group-snapshot-alpha/

https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/3476-volume-group-snapshot

Goals (aka. expected user outcomes)

Productise the volume group snapshots feature as tech preview have docs, testing as well as a feature gate to enable it in order for customers and partners to test it in advance.

Requirements (aka. Acceptance Criteria):

The feature should be graduated beta upstream to become TP in OCP. Tests and CI must pass and a feature gate should allow customers and partners to easily enable it. We should identify all OCP shipped CSI drivers that support this feature and configure them accordingly.

Use Cases (Optional):

1. As a storage vendor I want to have early access to the VolumeGroupSnapshot feature to test and validate my driver support.
2. As a backup vendor I want to have early access to the VolumeGroupSnapshot feature to test and validate my backup solution.
3. As a customer I want early access to test the VolumeGroupSnapshot feature in order to take consistent snapshots of my workloads that are relying on multiple PVs.

Out of Scope

CSI drivers development/support of this feature.

Background

Provide any additional context is needed to frame the feature. Initial completion during Refinement status.

Customer Considerations

Drivers must support this feature and enable it. Partners may need to change their operator and/or doc to support it.

Documentation Considerations

Document how to enable the feature, what this feature does and how to use it. Update the OCP driver's table to include this capability.

Interoperability Considerations

Can be leveraged by ODF and OCP virt, especially around backup and DR scenarios.

Feature Overview (aka. Goal Summary)  

Crun is GA as non default since OCP 4.14 . We want to make it as default in 4.18 while still supporting runc as non-default 

Benefits of Crun is covered here https://github.com/containers/crun 

FAQ.:  https://docs.google.com/document/d/1N7tik4HXTKsXS-tMhvnmagvw6TE44iNccQGfbL_-eXw/edit

***Note -> making Crun default does not means we will remove the support for runc nor we have any plans in foreseeable future to do that  

Feature Overview

This is Image mode on OpenShift. It uses the rpm-ostree native containers interface and not bootc but that is an implementation detail.

In the initial delivery of CoreOS Layering, it is required that administrators provide their own build environment to customize RHCOS images. That could be a traditional RHEL environment or potentially an enterprising administrator with some knowledge of OCP Builds could set theirs up on-cluster.

The primary virtue of an on-cluster build path is to continue using the cluster to manage the cluster. No external dependency, batteries-included.

On-cluster, automated RHCOS Layering builds are important for multiple reasons:

• One-click/one-command upgrades of OCP are very popular. Many customers may want to make one or just a few customizations but also want to keep that simplified upgrade experience. 
• Customers who only need to customize RHCOS temporarily (hotfix, driver test package, etc) will find off-cluster builds to be too much friction for one driver.
• One of OCP's virtues is that the platform and OS are developed, tested, and versioned together. Off-cluster building breaks that connection and leaves it up to the user to keep the OS up-to-date with the platform containers. We must make it easy for customers to add what they need and keep the OS image matched to the platform containers.

Goals & Requirements

• The goal of this feature is primarily to bring the 4.14 progress (OCPSTRAT-35) to a Tech Preview or GA level of support.
• Customers should be able to specify a Containerfile with their customizations and "forget it" as long as the automated builds succeed. If they fail, the admin should be alerted and pointed to the logs from the failed build.
  • The admin should then be able to correct the build and resume the upgrade.
• Intersect with the Custom Boot Images such that a required custom software component can be present on every boot of every node throughout the installation process including the bootstrap node sequence (example: out-of-box storage driver needed for root disk).
• Users can return a pool to an unmodified image easily.
• RHEL entitlements should be wired in or at least simple to set up (once).
• Parity with current features – including the current drain/reboot suppression list, CoreOS Extensions, and config drift monitoring.

Feature Overview

As a cluster admin for standalone OpenShift, I want to customize the prefix of the machine names created by CPMS due to company policies related to nomenclature. Implement the Control Plane Machine Set (CPMS) feature in OpenShift to support machine names where user can set custom names prefixes. Note the prefix will always be suffixed by "<5-chars>-<index>" as this is part of the CPMS internal design.

Acceptance Criteria

A new field called machineNamePrefix has been added to CPMS CR.
This field would allow the customer to specify a custom prefix for the machine names.
The machine names would then be generated using the format: <machineNamePrefix><5-chars><index>
Where:
<machineNamePrefix> is the custom prefix provided by the customer
<5-chars> is a random 5 character string (this is required and cannot be changed)
<index> represents the index of the machine (0, 1, 2, etc.)
Ensure that if the machineNamePrefix is changed, the operator reconciles and succeeds in rolling out the changes.

Feature Overview (aka. Goal Summary)  

Edge customers requiring computing on-site to serve business applications (e.g., point of sale, security & control applications, AI inference) are asking for a 2-node HA solution for their environments. Only two nodes at the edge, because the 3d node induces too much cost, but still they need HA for critical workload. To address this need, a 2+1 topology is introduced. It supports a small cheap arbiter node that can optionally be remote/virtual to reduce onsite HW cost. 

Goals (aka. expected user outcomes)

Support OpenShift on 2+1 topology, meaning two primary nodes with large capacity to run workload and control plan, and a third small “arbiter” node which ensure quorum. See requirements for more details.

Requirements (aka. Acceptance Criteria):

1. Co-located arbiter node -  3d node in same network/location with low latency network access, but the arbiter node is much smaller compared to the two main nodes. Target resource requirements for the arbiter node: 4 cores / 8 vcpu, 16G RAM, 120G disk (non-spinning), 1x1 GbE network ports, no BMC
2. OCP Virt fully functionally, incl. Live migration of VMs (assuming RWX CSI Driver is available)
3. Single Node outage is handled seamlessly
4. In case the arbiter node is down , a reboot/restart of the two remaining nodes has to work, i.e. the two remaining nodes re-gain quorum and spin-up the workload. 
5. Scale out  of the cluster by adding additional worker nodes should be possible
6. Transition the cluster into a regular 3 node compact cluster, e.g. by adding a new node as control plane node, then removing the arbiter node, should be possible
7. Regular workload should not be scheduled to the arbiter node (e.g by making it un-schedulabe, or introduce a new node role “arbiter”). Only essential control plane workload (etcd components) should run on the arbiter node. Non-essential control plan workload (i.e. router, registry, console, monitoring etc) should also not be scheduled to the arbiter nodded.
8. It must be possible to explicitly schedule additional workload to the arbiter node. That is important for 3d party solutions (e.g. storage provider) which also have  quorum based mechanisms.
9. must seamlessly integrate into existing installation/update mechanisms, esp. zero touch provisioning etc.
10. Added: ability to track OLA usage in the fleet of connected clusters via OCP telemetry data

Questions to Answer (Optional):

1. How to implement the scheduling restrictions to the arbiter node? New node role “arbiter”?
2. Can this be delivered in one release, or do we need to split, e.g. TechPreview + GA?

Out of Scope

1. Storage driver providing RWX shared storage
2. …

Background

Provide any additional context is needed to frame the feature.  Initial completion during Refinement status.

• Two node support is in high demand by telco, industrial and retail customers.
• VMWare supports a two node VSan solution: https://core.vmware.com/resource/vsan-2-node-cluster-guide
• Example edge hardware frequently used for edge deployments with a co-located small arbiter node: Dell PowerEdge XR4000z Server is an edge computing device that allows restaurants, retailers, and other small to medium businesses to set up local computing for data-intensive workloads. 

Customer Considerations

See requirements - there are two main groups of customers: co-located arbiter node, and remote arbiter node.

Documentation Considerations

1. Topology needs to be documented, esp. The requirements of the arbiter node.

Interoperability Considerations

1. OCP Virt needs to be explicitly tested on this scenario to support VM HA (live migration, restart on other node)

• https://spaces.redhat.com/display/PLUG/Edge+Sprint+Demos#EdgeSprintDemos-Sprint260

Feature Overview

Ability to install OpenShift on Nutanix with nodes having multiple NICs (multiple subnets) from IPI and for autoscaling with MachineSets.

Feature Overview

Support mapping OpenShift zones to vSphere host groups, in addition to vSphere clusters.

When defining zones for vSphere administrators can map regions to vSphere datacenters and zones to vSphere clusters.

There are use cases where vSphere clusters have only one cluster construct with all their ESXi hosts but the administrators want to divide the ESXi hosts in host groups. A common example is vSphere stretched clusters, where there is only one logical vSphere cluster but the ESXi nodes are distributed across to physical sites, and grouped by site in vSphere host groups.

In order for OpenShift to be able to distribute its nodes on vSphere matching the physical grouping of hosts, OpenShift zones have to be able to map to vSphere host groups too.

Requirements{}

• Users can define OpenShift zones mapping them to host groups at installation time (day 1)
• Users can use host groups as OpenShift zones post-installation (day 2)

Feature Overview (aka. Goal Summary)  

An elevator pitch (value statement) that describes the Feature in a clear, concise way.  Complete during New status.

<your text here>

Goals (aka. expected user outcomes)

The observable functionality that the user now has as a result of receiving this feature. Include the anticipated primary user type/persona and which existing features, if any, will be expanded. Complete during New status.

<your text here>

Requirements (aka. Acceptance Criteria):

A list of specific needs or objectives that a feature must deliver in order to be considered complete.  Be sure to include nonfunctional requirements such as security, reliability, performance, maintainability, scalability, usability, etc.  Initial completion during Refinement status.

<enter general Feature acceptance here>

Anyone reviewing this Feature needs to know which deployment configurations that the Feature will apply to (or not) once it's been completed.  Describe specific needs (or indicate N/A) for each of the following deployment scenarios. For specific configurations that are out-of-scope for a given release, ensure you provide the OCPSTRAT (for the future to be supported configuration) as well.

Use Cases (Optional):

Include use case diagrams, main success scenarios, alternative flow scenarios.  Initial completion during Refinement status.

<your text here>

Questions to Answer (Optional):

Include a list of refinement / architectural questions that may need to be answered before coding can begin.  Initial completion during Refinement status.

<your text here>

Out of Scope

High-level list of items that are out of scope.  Initial completion during Refinement status.

<your text here>

Background

Provide any additional context is needed to frame the feature.  Initial completion during Refinement status.

<your text here>

Customer Considerations

Provide any additional customer-specific considerations that must be made when designing and delivering the Feature.  Initial completion during Refinement status.

<your text here>

Documentation Considerations

Provide information that needs to be considered and planned so that documentation will meet customer needs.  If the feature extends existing functionality, provide a link to its current documentation. Initial completion during Refinement status.

<your text here>

Interoperability Considerations

Which other projects, including ROSA/OSD/ARO, and versions in our portfolio does this feature impact?  What interoperability test scenarios should be factored by the layered products?  Initial completion during Refinement status.

<your text here>

Feature Overview (aka. Goal Summary)  

An elevator pitch (value statement) that describes the Feature in a clear, concise way.  Complete during New status.

<your text here>

Goals (aka. expected user outcomes)

The observable functionality that the user now has as a result of receiving this feature. Include the anticipated primary user type/persona and which existing features, if any, will be expanded. Complete during New status.

<your text here>

Requirements (aka. Acceptance Criteria):

A list of specific needs or objectives that a feature must deliver in order to be considered complete.  Be sure to include nonfunctional requirements such as security, reliability, performance, maintainability, scalability, usability, etc.  Initial completion during Refinement status.

<enter general Feature acceptance here>

Anyone reviewing this Feature needs to know which deployment configurations that the Feature will apply to (or not) once it's been completed.  Describe specific needs (or indicate N/A) for each of the following deployment scenarios. For specific configurations that are out-of-scope for a given release, ensure you provide the OCPSTRAT (for the future to be supported configuration) as well.

Use Cases (Optional):

Include use case diagrams, main success scenarios, alternative flow scenarios.  Initial completion during Refinement status.

<your text here>

Questions to Answer (Optional):

Include a list of refinement / architectural questions that may need to be answered before coding can begin.  Initial completion during Refinement status.

<your text here>

Out of Scope

High-level list of items that are out of scope.  Initial completion during Refinement status.

<your text here>

Background

Provide any additional context is needed to frame the feature.  Initial completion during Refinement status.

<your text here>

Customer Considerations

Provide any additional customer-specific considerations that must be made when designing and delivering the Feature.  Initial completion during Refinement status.

<your text here>

Documentation Considerations

Provide information that needs to be considered and planned so that documentation will meet customer needs.  If the feature extends existing functionality, provide a link to its current documentation. Initial completion during Refinement status.

<your text here>

Interoperability Considerations

Which other projects, including ROSA/OSD/ARO, and versions in our portfolio does this feature impact?  What interoperability test scenarios should be factored by the layered products?  Initial completion during Refinement status.

<your text here>

Feature Overview (aka. Goal Summary)  

An elevator pitch (value statement) that describes the Feature in a clear, concise way.  Complete during New status.

K8s 1.31 introduces VolumeAttributesClass as beta (code in external provisioner). We should make it available to customers as tech preview.

VolumeAttributesClass allows PVC to be modified after their creation and while attached. There is as vast number of parameters that can be updated but the most popular is to change the QoS values. Parameters that can be changed depend on the driver used.

Goals (aka. expected user outcomes)

The observable functionality that the user now has as a result of receiving this feature. Include the anticipated primary user type/persona and which existing features, if any, will be expanded. Complete during New status.

Productise VolumeAttributesClass as TP in anticipation for GA. Customer can start testing VolumeAttributesClass.

Requirements (aka. Acceptance Criteria):

A list of specific needs or objectives that a feature must deliver in order to be considered complete.  Be sure to include nonfunctional requirements such as security, reliability, performance, maintainability, scalability, usability, etc.  Initial completion during Refinement status.

• Disabled by default
• put it under TechPreviewNoUpgrade
• make sure VolumeAttributeClass object is available in beta APIs
• enable the feature in external-provisioner and external-resizer at least in AWS EBS CSI driver, check the other drivers.
  • Add RBAC rules for these objects
• make sure we run its tests in one of TechPreviewNoUpgrade CI jobs (with hostpath CSI driver)
• reuse / add a job with AWS EBS CSI driver + tech preview.

Anyone reviewing this Feature needs to know which deployment configurations that the Feature will apply to (or not) once it's been completed.  Describe specific needs (or indicate N/A) for each of the following deployment scenarios. For specific configurations that are out-of-scope for a given release, ensure you provide the OCPSTRAT (for the future to be supported configuration) as well.

Use Cases (Optional):

Include use case diagrams, main success scenarios, alternative flow scenarios.  Initial completion during Refinement status.

As an OCP user, I want to change parameters of my existing PVC such as the QoS attributes.

Questions to Answer (Optional):

Include a list of refinement / architectural questions that may need to be answered before coding can begin.  Initial completion during Refinement status.

<your text here>

Out of Scope

High-level list of items that are out of scope.  Initial completion during Refinement status.

UI for TP

Background

Provide any additional context is needed to frame the feature.  Initial completion during Refinement status.

There's been some limitations and complains on the fact that PVC attributes are sealed after their creation avoiding customers to update them. This is particularly impacting for a specific QoS is set and the volume requirements are changing.

Customer Considerations

Provide any additional customer-specific considerations that must be made when designing and delivering the Feature.  Initial completion during Refinement status.

Customer should not use it in production atm.

Documentation Considerations

Provide information that needs to be considered and planned so that documentation will meet customer needs.  If the feature extends existing functionality, provide a link to its current documentation. Initial completion during Refinement status.

Document VolumeAttributesClass creation and how to update PVC. Mention any limitation. Mention it's tech preview no upgrade. Add drivers support if needed.

Interoperability Considerations

Which other projects, including ROSA/OSD/ARO, and versions in our portfolio does this feature impact?  What interoperability test scenarios should be factored by the layered products?  Initial completion during Refinement status.

Check which drivers support it for which parameters.

Feature Overview (aka. Goal Summary)  

A common concern with dealing with escalations/incidents in Managed OpenShift Hosted Control Planes is the resolution time incurred when the fix needs to be delivered in a component of the solution that ships within the OpenShift release payload. This is because OpenShift's release payloads:

•  Have a hotfix process that is customer/support-exception targeted rather than fleet targeted
• Can take weeks to be available for Managed OpenShift

This feature seeks to provide mechanisms that put the upper time boundary in delivering such fixes to match the current HyperShift Operator <24h expectation

Goals (aka. expected user outcomes)

• Hosted Control Plane fixes are delivered through Konflux builds
• No additional upgrade edges
• Release specific
• Adequate, fleet representative, automated testing coverage
• Reduced human interaction

Requirements (aka. Acceptance Criteria):

A list of specific needs or objectives that a feature must deliver in order to be considered complete.  Be sure to include nonfunctional requirements such as security, reliability, performance, maintainability, scalability, usability, etc.  Initial completion during Refinement status.

• Overriding Hosted Control Plane components can be done automatically once the PRs are ready and the affected versions have been properly identified
• Managed OpenShift Hosted Clusters have their Control Planes fix applied without requiring customer intervention and without workload disruption beyond what might already be incurred because of the incident it is solving
• Fix can be promoted through integration, stage and production canary with a good degree of observability

Anyone reviewing this Feature needs to know which deployment configurations that the Feature will apply to (or not) once it's been completed.  Describe specific needs (or indicate N/A) for each of the following deployment scenarios. For specific configurations that are out-of-scope for a given release, ensure you provide the OCPSTRAT (for the future to be supported configuration) as well.

Use Cases (Optional):

• Incident response when the engineering solution is partially or completely in the Hosted Control Plane side rather than in the HyperShift Operator

Out of Scope

• HyperShift Operator binary bundling

Background

Discussed previously during incident calls. Design discussion document

Customer Considerations

• Because the Managed Control Plane version does not change but it is overridden, customer visibility and impact should be limited as much as possible.

Documentation Considerations

SOP needs to be defined for:

• Requesting and approving the fleet wide fixes described above
• Building and delivering them
• Identifying clusters with deployed fleet wide fixes

Feature Overview

The OpenShift IPsec implementation will be enhanced for a growing set of enterprise use cases, and for larger scale deployments. 

Goals

The OpenShift IPsec implementation was originally built for purpose-driven use cases from telco NEPs, but was also be useful for a specific set of other customer use cases outside of that context.  As customer adoption grew and it was adopted by some of the largest (by number of cluster nodes) deployments in the field, it became obvious that some redesign is necessary in order to continue to deliver enterprise-grade IPsec, for both East-West and North-South traffic, and for some of our most-demanding customer deployments.  

Key enhancements include observability and blocked traffic across paths if IPsec encryption is not functioning properly.   

Requirements

Questions to answer…

Out of Scope

• Configuration of external-to-cluster IPsec endpoints for N-S IPsec. 

Background, and strategic fit

The OpenShift IPsec feature is fundamental to customer deployments for ensuring that all traffic between cluster nodes (East-West) and between cluster nodes and external-to-the-cluster entities that also are configured for IPsec (North-South) is encrypted by default.  This encryption must scale to the largest of deployments. 

Assumptions

Customer Considerations

• Customers require the option to use their own certificates or CA for IPsec. 
• Customers require observability of configuration (e.g. is the IPsec tunnel up and passing traffic)
• If the IPsec tunnel is not up or otherwise functioning, traffic across the intended-to-be-encrypted network path should be blocked. 

Documentation Considerations

Questions to be addressed:

• What educational or reference material (docs) is required to support this product feature? For users/admins? Other functions (security officers, etc)?
• Does this feature have doc impact?
• New Content, Updates to existing content, Release Note, or No Doc Impact
• If unsure and no Technical Writer is available, please contact Content Strategy.
• What concepts do customers need to understand to be successful in [action]?
• How do we expect customers will use the feature? For what purpose(s)?
• What reference material might a customer want/need to complete [action]?
• Is there source material that can be used as reference for the Technical Writer in writing the content? If yes, please link if available.
• What is the doc impact (New Content, Updates to existing content, or Release Note)?

Feature Overview (aka. Goal Summary)  

An elevator pitch (value statement) that describes the Feature in a clear, concise way.  Complete during New status.

<your text here>

Goals (aka. expected user outcomes)

The observable functionality that the user now has as a result of receiving this feature. Include the anticipated primary user type/persona and which existing features, if any, will be expanded. Complete during New status.

<your text here>

Requirements (aka. Acceptance Criteria):

A list of specific needs or objectives that a feature must deliver in order to be considered complete.  Be sure to include nonfunctional requirements such as security, reliability, performance, maintainability, scalability, usability, etc.  Initial completion during Refinement status.

<enter general Feature acceptance here>

Anyone reviewing this Feature needs to know which deployment configurations that the Feature will apply to (or not) once it's been completed.  Describe specific needs (or indicate N/A) for each of the following deployment scenarios. For specific configurations that are out-of-scope for a given release, ensure you provide the OCPSTRAT (for the future to be supported configuration) as well.

Use Cases (Optional):

Include use case diagrams, main success scenarios, alternative flow scenarios.  Initial completion during Refinement status.

<your text here>

Questions to Answer (Optional):

Include a list of refinement / architectural questions that may need to be answered before coding can begin.  Initial completion during Refinement status.

<your text here>

Out of Scope

High-level list of items that are out of scope.  Initial completion during Refinement status.

<your text here>

Background

Provide any additional context is needed to frame the feature.  Initial completion during Refinement status.

<your text here>

Customer Considerations

Provide any additional customer-specific considerations that must be made when designing and delivering the Feature.  Initial completion during Refinement status.

<your text here>

Documentation Considerations

Provide information that needs to be considered and planned so that documentation will meet customer needs.  If the feature extends existing functionality, provide a link to its current documentation. Initial completion during Refinement status.

<your text here>

Interoperability Considerations

Which other projects, including ROSA/OSD/ARO, and versions in our portfolio does this feature impact?  What interoperability test scenarios should be factored by the layered products?  Initial completion during Refinement status.

<your text here>

As a customer of self managed OpenShift or an SRE managing a fleet of OpenShift clusters I should be able to determine the progress and state of an OCP upgrade and only be alerted if the cluster is unable to progress. Support a cli-status command and status-API which can be used by cluster-admin to monitor the progress. status command/API should also contain data to alert users about potential issues which can make the updates problematic.

Feature Overview (aka. Goal Summary)  

Here are common update improvements from customer interactions on Update experience

1. Show nodes where pod draining is taking more time.
   Customers have to dig deeper often to find the nodes for further debugging. 
   The ask has been to bubble up this on the update progress window.
2. oc update status ?
   From the UI we can see the progress of the update. From oc cli we can see this from "oc get cvo"  
    But the ask is to show more details in a human-readable format.
   Know where the update has stopped. Consider adding at what run level it has stopped.
    

   oc get clusterversion
   NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
   
   version   4.12.0    True        True          16s     Working towards 4.12.4: 9 of 829 done (1% complete)
   

    

Documentation Considerations

Update docs for UX and CLI changes

Reference : https://docs.google.com/presentation/d/1cwxN30uno_9O8RayvcIAe8Owlds-5Wjr970sDxn7hPg/edit#slide=id.g2a2b8de8edb_0_22

Feature Overview (aka. Goal Summary)  

We need to maintain our dependencies across all the libraries we use in order to stay in compliance. 

Goals (aka. expected user outcomes)

The observable functionality that the user now has as a result of receiving this feature. Include the anticipated primary user type/persona and which existing features, if any, will be expanded. Complete during New status.

<your text here>

Requirements (aka. Acceptance Criteria):

A list of specific needs or objectives that a feature must deliver in order to be considered complete.  Be sure to include nonfunctional requirements such as security, reliability, performance, maintainability, scalability, usability, etc.  Initial completion during Refinement status.

<enter general Feature acceptance here>

Anyone reviewing this Feature needs to know which deployment configurations that the Feature will apply to (or not) once it's been completed.  Describe specific needs (or indicate N/A) for each of the following deployment scenarios. For specific configurations that are out-of-scope for a given release, ensure you provide the OCPSTRAT (for the future to be supported configuration) as well.

Use Cases (Optional):

Include use case diagrams, main success scenarios, alternative flow scenarios.  Initial completion during Refinement status.

<your text here>

Questions to Answer (Optional):

Include a list of refinement / architectural questions that may need to be answered before coding can begin.  Initial completion during Refinement status.

<your text here>

Out of Scope

High-level list of items that are out of scope.  Initial completion during Refinement status.

<your text here>

Background

Provide any additional context is needed to frame the feature.  Initial completion during Refinement status.

<your text here>

Customer Considerations

Provide any additional customer-specific considerations that must be made when designing and delivering the Feature.  Initial completion during Refinement status.

<your text here>

Documentation Considerations

Provide information that needs to be considered and planned so that documentation will meet customer needs.  If the feature extends existing functionality, provide a link to its current documentation. Initial completion during Refinement status.

<your text here>

Interoperability Considerations

Which other projects, including ROSA/OSD/ARO, and versions in our portfolio does this feature impact?  What interoperability test scenarios should be factored by the layered products?  Initial completion during Refinement status.

<your text here>

Feature Overview (aka. Goal Summary)  

The ability in OpenShift to create trust and directly consume access tokens issued by external OIDC Authentication Providers using an authentication approach similar to upstream Kubernetes.

BYO Identity will help facilitate CLI only workflows and capabilities of the Authentication Provider (such as Keycloak, Dex, Azure AD) similar to upstream Kubernetes. 

Goals (aka. expected user outcomes)

Ability in OpenShift to provide a direct, pluggable Authentication workflow such that the OpenShift/K8s API server can consume access tokens issued by external OIDC identity providers. Kubernetes provides this integration as described here. Customer/Users can then configure their IDPs to support the OIDC protocols and workflows they desire such as Client credential flow.

OpenShift OAuth server is still available as default option, with the ability to tune in the external OIDC provider as a Day-2 configuration.

Requirements (aka. Acceptance Criteria):

1. The customer should be able to tie into RBAC functionality, similar to how it is closely aligned with OpenShift OAuth 

2.  

Use Cases (Optional):

1. As a customer, I would like to integrate my OIDC Identity Provider directly with the OpenShift API server.
2. As a customer in multi-cluster cloud environment, I have both K8s and non-K8s clusters using my IDP and hence I need seamless authentication directly to the OpenShift/K8sAPI using my Identity Provider 
3.  

Questions to Answer (Optional):

Include a list of refinement / architectural questions that may need to be answered before coding can begin.  Initial completion during Refinement status.

Out of Scope

High-level list of items that are out of scope.  Initial completion during Refinement status.

Background

Provide any additional context is needed to frame the feature.  Initial completion during Refinement status.

Customer Considerations

Provide any additional customer-specific considerations that must be made when designing and delivering the Feature.  Initial completion during Refinement status.

Documentation Considerations

Provide information that needs to be considered and planned so that documentation will meet customer needs.  Initial completion during Refinement status.

Interoperability Considerations

Which other projects and versions in our portfolio does this feature impact?  What interoperability test scenarios should be factored by the layered products?  Initial completion during Refinement status.

Upstream K8s deprecated PodSecurityPolicy and replaced it with a new built-in admission controller that enforces the Pod Security Standards (See here for the motivations for deprecation).] There is an OpenShift-specific dedicated pod admission system called Security Context Constraints. Our aim is to keep the Security Context Constraints pod admission system while also allowing users to have access to the Kubernetes Pod Security Admission. 

With OpenShift 4.11, we are turned on the Pod Security Admission with global "privileged" enforcement. Additionally we set the "restricted" profile for warnings and audit. This configuration made it possible for users to opt-in their namespaces to Pod Security Admission with the per-namespace labels. We also introduced a new mechanism that automatically synchronizes the Pod Security Admission "warn" and "audit" labels.

With OpenShift 4.15, we intend to move the global configuration to enforce the "restricted" pod security profile globally. With this change, the label synchronization mechanism will also switch into a mode where it synchronizes the "enforce" Pod Security Admission label rather than the "audit" and "warn". 

Feature Overview (aka. Goal Summary)  

Implement Migration core for MAPI to CAPI for AWS

• This feature covers the design and implementation of converting from using the Machine API (MAPI) to Cluster API (CAPI) for AWS
• This Design investigates possible solutions for AWS
• Once AWS shim/sync layer is implemented use the architecture for other clouds in phase-2 & phase 3

Acceptance Criteria

When customers use CAPI, There must be no negative effect to switching over to using CAPI .  Seamless migration of Machine resources. the fields in MAPI/CAPI should reconcile from both CRDs.

Feature Overview (aka. Goal Summary)  

As a cluster administrator, I want to use Karpenter on an OpenShift cluster running in AWS to scale nodes instead of Cluster Autoscalar(CAS). I want to automatically manage heterogeneous compute resources in my OpenShift cluster without the additional manual task of managing node pools. Additional features I want are:

• Reducing cloud costs through instance selection and scaling/descaling
• Support GPUs, spot instances, mixed compute types and other compute types.
• Automatic node lifecycle management and upgrades

This feature covers the work done to integrate upstream Karpenter 1.x with ROSA HCP. This eliminates the need for manual node pool management while ensuring cost-effective compute selection for workloads. Red Hat manages the node lifecycle and upgrades.

The feature will be rolled out with ROSA (AWS) since it has more mature Karpenter ecosystem, followed by ARO (Azure) implementation(check OCPSTRAT-1498)

Goals (aka. expected user outcomes)

1. Run Karpenter in management cluster and disable CAS
2. Automate node provisioning in workload cluster
3. automate lifecycle management  in workload cluster
4. Reduce cost in heterogenous compute workloads

https://docs.google.com/document/d/1ID_IhXPpYY4K3G_wa1MYJxOb3yz5FYoOj3ONSkEDsZs/edit?tab=t.0#heading=h.yvv1wy2g0utk

Requirements (aka. Acceptance Criteria):

As a cluster-admin or SRE I should be able to configure Karpenter with OCP on AWS. Both cli and UI should enable users to configure Karpenter and disable CAS.

1. Run Karpenter in management cluster and disable CAS
2. OCM API 
   • Enable/Disable Cluster autoscaler
   • Enable/disable AutoNode feature
   • New ARN role configuration for Karpenter
   • Optional: New managed policy or integration with existing nodepool role permissions
3. Expose NodeClass/Nodepool resources to users. 
4. secure node provisioning and management, machine approval system for Karpenter instances
5. HCP Karpenter cleanup/deletion support
6. ROSA CAPI fields to enable/disable/configure Karpenter
7. Write end-to-end tests for karpenter running on ROSA HCP

Anyone reviewing this Feature needs to know which deployment configurations that the Feature will apply to (or not) once it's been completed.  Describe specific needs (or indicate N/A) for each of the following deployment scenarios. For specific configurations that are out-of-scope for a given release, ensure you provide the OCPSTRAT (for the future to be supported configuration) as well.

Use Cases (Optional):

Include use case diagrams, main success scenarios, alternative flow scenarios.  Initial completion during Refinement status.

<your text here>

Questions to Answer (Optional):

Include a list of refinement / architectural questions that may need to be answered before coding can begin.  Initial completion during Refinement status.

<your text here>

Out of Scope

High-level list of items that are out of scope.  Initial completion during Refinement status.

• Supporting this feature in Standalone OCP/self-hosted HCP/ROSA classic
• Creating a multi-provider cost/pricing operator compatible with CAPI is beyond the scope of this Feature. That may take more time.
•  

Background

Provide any additional context is needed to frame the feature.  Initial completion during Refinement status.

• Karpenter.sh is an open-source node provisioning project built for Kubernetes. It is designed to simplify Kubernetes infrastructure by automatically launching and terminating nodes based on the needs of your workloads. Karpenter can help you to reduce costs, improve performance, and simplify operations.
• Karpenter works by observing the unscheduled pods in your cluster and launching new nodes to accommodate them. Karpenter can also terminate nodes that are no longer needed, which can help you save money on infrastructure costs.
• Karpenter architecture has a Karpenter-core and Karpenter-provider as components. 
  The core has AWS code which does the resource calculation to reduce the cost by re-provisioning new nodes.

Customer Considerations

Provide any additional customer-specific considerations that must be made when designing and delivering the Feature.  Initial completion during Refinement status.

<your text here>

Documentation Considerations

Provide information that needs to be considered and planned so that documentation will meet customer needs.  If the feature extends existing functionality, provide a link to its current documentation. Initial completion during Refinement status.

• Migration guides from using CAS to Karpenter
• Performance testing to compare CAS vs Karpenter on ROSA HCP
• API documentation for NodePool and EC2NodeClass configuration

Interoperability Considerations

Which other projects, including ROSA/OSD/ARO, and versions in our portfolio does this feature impact?  What interoperability test scenarios should be factored by the layered products?  Initial completion during Refinement status.

<your text here>

Goal Summary

This feature aims to make sure that the HyperShift operator and the control-plane it deploys uses Managed Service Identities (MSI) and have access to scoped credentials (also via access to AKS's image gallery potentially). Additionally, for operators deployed in customers account (system components), they would be scoped with Azure workload identities. 

Feature Overview

• An assistant to help developers in ODC edit configuration YML files

Goals

• Perform an architectural spike to better assess feasibility and value of pursuing this further

Requirements

•  Refinement DOC

(Optional) Use Cases

This Section:

• Main success scenarios - high-level user stories
• Alternate flow/scenarios - high-level user stories
• ...

Questions to answer…

• Is there overlap with what other teams at RH are already planning? 

Out of Scope

• …

Background, and strategic fit

This Section: What does the person writing code, testing, documenting need to know? What context can be provided to frame this feature.

Assumptions

• ...

Customer Considerations

• More details in the outcome parent RHDP-985

Documentation Considerations

Questions to be addressed:

• What educational or reference material (docs) is required to support this product feature? For users/admins? Other functions (security officers, etc)?
• Does this feature have doc impact?
• New Content, Updates to existing content, Release Note, or No Doc Impact
• If unsure and no Technical Writer is available, please contact Content Strategy.
• What concepts do customers need to understand to be successful in [action]?
• How do we expect customers will use the feature? For what purpose(s)?
• What reference material might a customer want/need to complete [action]?
• Is there source material that can be used as reference for the Technical Writer in writing the content? If yes, please link if available.
• What is the doc impact (New Content, Updates to existing content, or Release Note)?

Epic Goal

This epic tracks any part of our codebase / solutions we implemented taking shortcuts.
Whenever a shortcut is taken, we should add a story here not to forget to improve it in a safer and more maintainabile way.

Why is this important?

maintanability and debuggability, and in general fighting the technical debt, is critical to keep velocity and ensure overall high quality

Scenarios

1. N/A

Acceptance Criteria

• depends on the specific card

Dependencies (internal and external)

• depends on the specific card

Previous Work (Optional):

https://issues.redhat.com/browse/CNF-796
https://issues.redhat.com/browse/CNF-1479 
https://issues.redhat.com/browse/CNF-2134
https://issues.redhat.com/browse/CNF-6745
https://issues.redhat.com/browse/CNF-8036
https://issues.redhat.com/browse/CNF-9566 

 Done Checklist

• CI - CI is running, tests are automated and merged.
• Release Enablement <link to Feature Enablement Presentation>
• DEV - Upstream code and tests merged: <link to meaningful PR or GitHub Issue>
• DEV - Upstream documentation merged: <link to meaningful PR or GitHub Issue>
• DEV - Downstream build attached to advisory: <link to errata>
• QE - Test plans in Polarion: <link or reference to Polarion>
• QE - Automated tests merged: <link or reference to automated tests>
• DOC - Downstream documentation merged: <link to meaningful PR>

Capture the necessary accidental work to get CI / Konflux unstuck during the 4.19 cycle

An epic we can duplicate for each release to ensure we have a place to catch things we ought to be doing regularly but can tend to fall by the wayside.

Goal

• ...

Why is this important?

• …

Scenarios

1. ...

Acceptance Criteria

• Dev - Has a valid enhancement if necessary
• CI - MUST be running successfully with tests automated
• QE - covered in Polarion test plan and tests implemented
• Release Technical Enablement - Must have TE slides
• ...

Dependencies (internal and external)

1. ...

Previous Work (Optional):

1. …

Open questions:

1. …

Done Checklist

• CI - CI is running, tests are automated and merged.
• Release Technical Enablement <link to Feature Enablement Presentation>
• DEV - Upstream documentation merged: <link to meaningful PR or GitHub Issue>
• DEV - Enhancement merged: <link to meaningful PR or GitHub Issue>
• QE - Test plans in Polarion: <link or reference to Polarion>
• QE - Automated tests merged: <link or reference to automated tests>
• DOC - Downstream documentation merged: <link to meaningful PR>

These are items that the team has prioritized to address in 4.18.

This epic has been repurposed for handling bugs and issues related to DataImage api ( see comments by Zane and slack discussion below ). Some issues have already been added, will add more issues to improve the stability and reliability of this feature.

Reference links :
Issue opened for IBIO : https://issues.redhat.com/browse/OCPBUGS-43330
Slack discussion threads :
https://redhat-internal.slack.com/archives/CFP6ST0A3/p1729081044547689?thread_ts=1728928990.795199&cid=CFP6ST0A3
https://redhat-internal.slack.com/archives/C0523LQCQG1/p1732110124833909?thread_ts=1731660639.803949&cid=C0523LQCQG1

Tracking all things Konflux related for the Metal Platform Team
Full enablement should happen during OCP 4.19 development cycle

Reference: https://issues.redhat.com/browse/KONFLUX-1611?filter=-1

Currently, the monitoring stack is configured using a configmap. In OpenShift though the best practice is to configure operators using custom resources.

Why this matters

• We can add [cross]validation rules to CRD fields to avoid misconfigurations
• End users get a much faster feedback loop. No more applying the config and scanning logs if things don't look right. The API server will give immediate feedback
• Organizational users (such as ACM) can manage a single resource and observe its status

To start the effort we should create a feature gate behind which we can start implementing a CRD config approach. This allows us to iterate in smaller increments without having to support full feature parity with the config map from the start. We can start small and add features as they evolve.

One proposal for a minimal DoD was:

• We have a feature gate
• We have outlined our idea and approach in an enhancement proposal. This does not have to be complete, just outline how we intend to implement this. OpenShift members have reviewed this and given their general approval. The OEP does not need to be complete or merged,
• We have CRD scaffolding that CVO creates and CMO watches
• We have a clear idea for a migration path. Even with a feature gate in place we may not simply switch config mechanisms, i.e. we must have a mechanism to merge settings from the config maps and CR, with the CR taking precedence.
• We have at least one or more fields, CMO can act upon. For example
  • a bool field telling CMO to use the config map for configuration
  • ...

Feature parity should be planned in one or more separate epics.

The history of this epic starts with this PR which triggered a lengthy conversation around the workings of the image  API with respect to importing imagestreams  images as single vs manifestlisted. The imagestreams today by default have the `importMode` flag set to `Legacy` to avoid breaking behavior of existing clusters in the field. This makes sense for single arch clusters deployed with a single  arch payload, but when users migrate to use the multi payload, more often than not, their intent is to add nodes of other architecture types. When this happens - it gives rise to problems when using imagestreams with the default behavior of importing a single manifest image. The oc commands do have a new flag to toggle the importMode, but this breaks functionality of existing users who just want to create an imagestream and use it with existing commands.

There was a discussion with David Eads and other staff engineers and it was decided that the approach to be taken is to default imagestreams' importMode to `preserveOriginal` if the cluster is installed with/ upgraded to a multi payload. So a few things need to happen to achieve this:

• CVO would need to expose a field in the status section indicative of the type of payload in the cluster (single vs multi)
• cluster-openshift-apiserver-operator would read this field and add it to the apiserver configmap. openshift-apiserver would use this value to determine the setting of importMode value.
• Document clearly that the behavior of imagestreams in a cluster with multi payload is different from the traditional single payload

Some open questions:

• What happens to existing imagestreams on upgrades
• How do we handle CVO managed imagestreams (IMO, CVO managed imagestreams should always set importMode to preserveOriginal as the images are associated with the payload)

This is a container Epic for tasks which we know need to be done for Tech Preview but which we don't intend to do now. It needs to be groomed before it is useful for planning.

Epic Goal*

What is our purpose in implementing this?  What new capability will be available to customers?

Why is this important? (mandatory)

What are the benefits to the customer or Red Hat?   Does it improve security, performance, supportability, etc?  Why is work a priority?

Scenarios (mandatory) 

Provide details for user scenarios including actions to be performed, platform specifications, and user personas.  

Dependencies (internal and external) (mandatory)

What items must be delivered by other teams/groups to enable delivery of this epic. 

Contributing Teams(and contacts) (mandatory) 

Our expectation is that teams would modify the list below to fit the epic. Some epics may not need all the default groups but what is included here should accurately reflect who will be involved in delivering the epic.

• Development - 
• Documentation -
• QE - 
• PX - 
• Others -

Acceptance Criteria (optional)

Provide some (testable) examples of how we will know if we have achieved the epic goal.  

Drawbacks or Risk (optional)

Reasons we should consider NOT doing this such as: limited audience for the feature, feature will be superseded by other work that is planned, resulting feature will introduce substantial administrative complexity or user confusion, etc.

Done - Checklist (mandatory)

The following points apply to all epics and are what the OpenShift team believes are the minimum set of criteria that epics should meet for us to consider them potentially shippable. We request that epic owners modify this list to reflect the work to be completed in order to produce something that is potentially shippable.

• CI Testing - Tests are merged and completing successfully
• Documentation - Content development is complete.
• QE - Test scenarios are written and executed successfully.
• Technical Enablement - Slides are complete (if requested by PLM)
• Other 

OCP/Telco Definition of Done
Epic Template descriptions and documentation.

<--- Cut-n-Paste the entire contents of this description into your new Epic --->

Epic Goal

• We want to remove official support for UPI and IPI support for Alibaba Cloud provider. Going forward, we are recommending installations on Alibaba Cloud with either external platform or agnostic platform installation method.

Why is this important?

An elevator pitch (value statement) that describes the Feature in a clear, concise way.  Complete during New status.

We have decided to remove IPI and UPI support for Alibaba Cloud, which until recently has been in Tech Preview due to the following reasons:

(1) Low customer interest of using Openshift on Alibaba Cloud

(2) Removal of Terraform usage

(3) MAPI to CAPI migration

(4) CAPI adoption for installation (Day 1) in OpenShift 4.16 and Day 2 (post-OpenShift 4.16)

Scenarios

Impacted areas based on CI:

alibaba-cloud-csi-driver/openshift-alibaba-cloud-csi-driver-release-4.16.yaml
alibaba-disk-csi-driver-operator/openshift-alibaba-disk-csi-driver-operator-release-4.16.yaml
cloud-provider-alibaba-cloud/openshift-cloud-provider-alibaba-cloud-release-4.16.yaml
cluster-api-provider-alibaba/openshift-cluster-api-provider-alibaba-release-4.16.yaml
cluster-cloud-controller-manager-operator/openshift-cluster-cloud-controller-manager-operator-release-4.16.yaml
machine-config-operator/openshift-machine-config-operator-release-4.16.yaml

Acceptance Criteria

• CI - MUST be running successfully with tests automated
• Release Technical Enablement - Provide necessary release enablement details and documents.
• ...

Dependencies (internal and external)

1. ...

Previous Work (Optional):

1. …

Open questions::

1. …

Done Checklist

• CI - CI jobs are removed
• Release Enablement <link to Feature Enablement Presentation>
• DEV - Upstream code and tests merged: <link to meaningful PR or GitHub Issue>
• DEV - Upstream documentation merged: <link to meaningful PR or GitHub Issue>
• DEV - Downstream build attached to advisory: <link to errata>
• QE - Test plans in Polarion: <link or reference to Polarion>
• QE - Automated tests merged: <link or reference to automated tests>
• DOC - Downstream documentation merged: <link to meaningful PR>

• https://github.com/openshift/cloud-credential-operator/pull/792

Test is failing due to oddness with oc adm logs.

We think it is related to PodLogsQuery feature that went into 1.32.