[goal] Deprecate the JSON formatted policy file

As per the community goal of migrating the policy file
the format from JSON to YAML[1], we need to do two things:

1. Change the default value of '[oslo_policy] policy_file''
config option from 'policy.json' to 'policy.yaml' with
upgrade checks.

2. Deprecate the JSON formatted policy file on the project side
via warning in doc and releasenotes.

Also replace policy.json to policy.yaml ref from doc and tests.

[1]https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html

Change-Id: I595b8c7cad7bb02486c8f0dd29fdc40789fc8e50

devstack/settings

@@ -5,7 +5,7 @@ ZAQAR_DIR=$DEST/zaqar
 ZAQARCLIENT_DIR=$DEST/python-zaqarclient
 ZAQAR_CONF_DIR=/etc/zaqar
 ZAQAR_CONF=$ZAQAR_CONF_DIR/zaqar.conf
-ZAQAR_POLICY_CONF=$ZAQAR_CONF_DIR/policy.json
+ZAQAR_POLICY_CONF=$ZAQAR_CONF_DIR/policy.yaml
 ZAQAR_UWSGI_CONF=$ZAQAR_CONF_DIR/uwsgi.conf
 ZAQAR_UWSGI_MASTER_PIDFILE=/tmp/uwsgizaqarmasterprocess.pid
 ZAQAR_API_LOG_DIR=/var/log/zaqar

doc/source/admin/subscription_confirm.rst

@@ -29,14 +29,14 @@ Webhook
    like: message_pipeline = zaqar.notification.notifier
 
 1. Set the config option "require_confirmation" and add the policy to the
-policy.json file. Then restart Zaqar-wsgi service::
+policy.yaml file. Then restart Zaqar-wsgi service::
 
     In the config file:
     [notification]
     require_confirmation = True
 
-    In the policy.json file:
-    "subscription:confirm": "",
+    In the policy.yaml file:
+    "subscription:confirm": ""
 
 2. Create a subscription.
 
@@ -244,8 +244,8 @@ unsubscribe confirmation email content, including topic, body and sender too::
                                               body:'You have unsubscribed successfully to the queue: {0}. This queue belongs to project: {1}. To resubscribe this subscription, click or visit this link below: {2}',\
                                               sender:Zaqar Notifications <no-reply@openstack.org>
 
-    In the policy.json file:
-    "subscription:confirm": "",
+    In the policy.yaml file:
+    "subscription:confirm": ""
 
 2. Create a subscription.
 For email confirmation, you should create a subscription like this::

doc/source/configuration/configuring.rst

@@ -57,6 +57,15 @@ Permission control options in each API version:
   * RBAC policy options: ``policy_default_rule``, ``policy_dirs``,
     ``policy_file`` which controls the permissions to access each type of
     functionality for different types of users.
+
+    .. warning::
+
+       JSON formatted policy file is deprecated since Zaqar 12.0.0 (Wallaby).
+       This `oslopolicy-convert-json-to-yaml`__ tool will migrate your existing
+       JSON-formatted policy file to YAML in a backward-compatible way.
+
+    .. __: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html
+
   * ``secret_key`` option which defines a secret key to use for signing
     special URLs. These are called pre-signed URLs and give temporary
     permissions to outsiders of the system.

lower-constraints.txt

@@ -20,18 +20,18 @@ msgpack==1.0.0
 os-api-ref==1.4.0
 os-client-config==1.28.0
 oslo.cache==1.26.0
-oslo.config==5.2.0
+oslo.config==6.8.0
 oslo.context==2.19.2
 oslo.db==6.0.0
 oslo.i18n==3.15.3
 oslo.log==3.36.0
 oslo.messaging==5.29.0
 oslo.middleware==3.31.0
-oslo.policy==1.30.0
+oslo.policy==3.6.0
 oslo.reports==1.18.0
 oslo.serialization==2.18.0
 oslo.service==1.24.0
-oslo.upgradecheck==0.1.0
+oslo.upgradecheck==1.3.0
 oslo.utils==3.33.0
 oslotest==3.2.0
 osprofiler==1.4.0
@@ -44,12 +44,12 @@ python-memcached==1.56
 python-subunit==1.0.0
 python-swiftclient==3.2.0
 pytz==2013.6
-PyYAML==3.13
+PyYAML==5.1
 redis==3.0.0
-requests==2.14.2
+requests==2.25.0
 requestsexceptions==1.2.0
 restructuredtext-lint==1.1.1
-rfc3986==0.3.1
+rfc3986==1.2.0
 SQLAlchemy==1.3.19
 sqlalchemy-migrate==0.11.0
 stestr==2.0.0

releasenotes/notes/deprecate-json-formatted-policy-file-f2abc160715c3f9b.yaml

@@ -0,0 +1,20 @@
+---
+upgrade:
+  - |
+    The default value of ``[oslo_policy] policy_file`` config option has
+    been changed from ``policy.json`` to ``policy.yaml``.
+    Operators who are utilizing customized or previously generated
+    static policy JSON files (which are not needed by default), should
+    generate new policy files or convert them in YAML format. Use the
+    `oslopolicy-convert-json-to-yaml
+    <https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html>`_
+    tool to convert a JSON to YAML formatted policy file in
+    backward compatible way.
+deprecations:
+  - |
+    Use of JSON policy files was deprecated by the ``oslo.policy`` library
+    during the Victoria development cycle. As a result, this deprecation is
+    being noted in the Wallaby cycle with an anticipated future removal of support
+    by ``oslo.policy``. As such operators will need to convert to YAML policy
+    files. Please see the upgrade notes for details on migration of any
+    custom policy files.

requirements.txt

@@ -16,7 +16,7 @@ python-swiftclient>=3.2.0 # Apache-2.0
 WebOb>=1.7.1 # MIT
 stevedore>=1.20.0 # Apache-2.0
 oslo.cache>=1.26.0 # Apache-2.0
-oslo.config>=5.2.0 # Apache-2.0
+oslo.config>=6.8.0 # Apache-2.0
 oslo.context>=2.19.2 # Apache-2.0
 oslo.db>=6.0.0 # Apache-2.0
 oslo.i18n>=3.15.3 # Apache-2.0
@@ -24,11 +24,11 @@ oslo.log>=3.36.0 # Apache-2.0
 oslo.messaging>=5.29.0 # Apache-2.0
 oslo.reports>=1.18.0 # Apache-2.0
 oslo.serialization!=2.19.1,>=2.18.0 # Apache-2.0
-oslo.upgradecheck>=0.1.0 # Apache-2.0
+oslo.upgradecheck>=1.3.0 # Apache-2.0
 oslo.utils>=3.33.0 # Apache-2.0
-oslo.policy>=1.30.0 # Apache-2.0
+oslo.policy>=3.6.0 # Apache-2.0
 osprofiler>=1.4.0 # Apache-2.0
 SQLAlchemy!=1.1.5,!=1.1.6,!=1.1.7,!=1.1.8,>=1.3.19 # MIT
 autobahn>=20.7.1 # MIT License
-requests>=2.14.2 # Apache-2.0
+requests>=2.25.0 # Apache-2.0
 futurist>=1.2.0 # Apache-2.0

zaqar/cmd/status.py

@@ -15,6 +15,7 @@
 import sys
 
 from oslo_config import cfg
+from oslo_upgradecheck import common_checks
 from oslo_upgradecheck import upgradecheck
 
 from zaqar.i18n import _
@@ -28,11 +29,6 @@ class Checks(upgradecheck.UpgradeCommands):
     and added to _upgrade_checks tuple.
     """
 
-    def _check_placeholder(self):
-        # This is just a placeholder for upgrade checks, it should be
-        # removed when the actual checks are added
-        return upgradecheck.Result(upgradecheck.Code.SUCCESS)
-
     # The format of the check functions is to return an
     # oslo_upgradecheck.upgradecheck.Result
     # object with the appropriate
@@ -41,8 +37,8 @@ class Checks(upgradecheck.UpgradeCommands):
     # in the returned Result's "details" attribute. The
     # summary will be rolled up at the end of the check() method.
     _upgrade_checks = (
-        # In the future there should be some real checks added here
-        (_('Placeholder'), _check_placeholder),
+        (_('policy File JSON to YAML Migration'),
+         (common_checks.check_policy_json, {'conf': cfg.CONF})),
     )
 
 

zaqar/tests/base.py

@@ -57,6 +57,7 @@ class TestBase(testtools.TestCase):
             self.conf = self.load_conf(self.config_file)
         else:
             self.conf = cfg.ConfigOpts()
+            self.conf(args=[], project='zaqar')
 
         self.conf.register_opts(default.ALL_OPTS)
         self.conf.register_opts(drivers.ALL_OPTS,

zaqar/tests/etc/policy.json

@@ -1,3 +0,0 @@
-{
-    "default": "rule:admin_or_owner"
-}

zaqar/tests/etc/policy.yaml

@@ -0,0 +1,4 @@
+# WARNING: Below rules are either deprecated rules
+# or extra rules in policy file, it is strongly
+# recommended to switch to new rules.
+"default": "rule:admin_or_owner"

zaqar/tests/helpers.py

@@ -288,7 +288,7 @@ def override_mongo_conf(conf_file, test):
             parser.set(section, 'uri', test_mongo_url)
         if not parser.has_section('oslo_policy'):
             parser.add_section('oslo_policy')
-        parser.set('oslo_policy', 'policy_file', test.conf_path('policy.json'))
+        parser.set('oslo_policy', 'policy_file', test.conf_path('policy.yaml'))
         fd, path = tempfile.mkstemp()
         conf_fd = os.fdopen(fd, 'w')
         try:

zaqar/tests/unit/cmd/test_status.py

@@ -1,30 +0,0 @@
-# Copyright (c) 2018 NEC, Corp.
-#
-#    Licensed under the Apache License, Version 2.0 (the "License"); you may
-#    not use this file except in compliance with the License. You may obtain
-#    a copy of the License at
-#
-#         http://www.apache.org/licenses/LICENSE-2.0
-#
-#    Unless required by applicable law or agreed to in writing, software
-#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
-#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
-#    License for the specific language governing permissions and limitations
-#    under the License.
-
-from oslo_upgradecheck.upgradecheck import Code
-
-from zaqar.cmd import status
-from zaqar.tests import base
-
-
-class TestUpgradeChecks(base.TestBase):
-
-    def setUp(self):
-        super(TestUpgradeChecks, self).setUp()
-        self.cmd = status.Checks()
-
-    def test__check_placeholder(self):
-        check_result = self.cmd._check_placeholder()
-        self.assertEqual(
-            Code.SUCCESS, check_result.code)

zaqar/transport/acl.py

@@ -16,12 +16,20 @@
 
 import functools
 
+from oslo_config import cfg
+from oslo_policy import opts
 from oslo_policy import policy
 
 from zaqar.common import policies
 
 ENFORCER = None
 
+# TODO(gmann): Remove setting the default value of config policy_file
+# once oslo_policy change the default value to 'policy.yaml'.
+# https://github.com/openstack/oslo.policy/blob/a626ad12fe5a3abd49d70e3e5b95589d279ab578/oslo_policy/opts.py#L49
+DEFAULT_POLICY_FILE = 'policy.yaml'
+opts.set_defaults(cfg.CONF, DEFAULT_POLICY_FILE)
+
 
 def setup_policy(conf):
     global ENFORCER