[pull] main from python:main

Doc/library/dialog.rst

@@ -217,9 +217,19 @@ These do not emulate the native look-and-feel of the platform.
 .. note::  The *FileDialog* class should be subclassed for custom event
    handling and behaviour.
 
-.. class:: FileDialog(master, title=None)
+.. class:: FileDialog(master, title=None, *, use_ttk=True)
 
    Create a basic file selection dialog.
+   Its layout -- a filter entry, side-by-side directory and file lists, and a
+   selection entry -- follows the classic Motif file selection dialog.
+   When *use_ttk* is true (the default), the dialog is built from the themed
+   :mod:`tkinter.ttk` widgets; when false, from the classic :mod:`tkinter`
+   widgets.
+
+   .. versionchanged:: next
+      The dialog is now built from the themed :mod:`tkinter.ttk` widgets by
+      default, instead of the classic :mod:`tkinter` widgets.
+      Added the *use_ttk* parameter.
 
    .. method:: cancel_command(event=None)
 
@@ -281,21 +291,27 @@ These do not emulate the native look-and-feel of the platform.
       Update the current file selection to *file*.
 
 
-.. class:: LoadFileDialog(master, title=None)
+.. class:: LoadFileDialog(master, title=None, *, use_ttk=True)
 
    A subclass of FileDialog that creates a dialog window for selecting an
    existing file.
 
+   .. versionchanged:: next
+      Added the *use_ttk* parameter.
+
    .. method:: ok_command()
 
       Test that a file is provided and that the selection indicates an
       already existing file.
 
-.. class:: SaveFileDialog(master, title=None)
+.. class:: SaveFileDialog(master, title=None, *, use_ttk=True)
 
    A subclass of FileDialog that creates a dialog window for selecting a
    destination file.
 
+   .. versionchanged:: next
+      Added the *use_ttk* parameter.
+
    .. method:: ok_command()
 
       Test whether or not the selection points to a valid file that is not a

Doc/library/imaplib.rst

@@ -16,7 +16,7 @@
 This module defines three classes, :class:`IMAP4`, :class:`IMAP4_SSL` and
 :class:`IMAP4_stream`, which encapsulate a connection to an IMAP4 server and
 implement a large subset of the IMAP4rev1 client protocol as defined in
-:rfc:`2060`. It is backward compatible with IMAP4 (:rfc:`1730`) servers, but
+:rfc:`3501`. It is backward compatible with IMAP4 (:rfc:`1730`) servers, but
 note that the ``STATUS`` command is not supported in IMAP4.
 
 .. include:: ../includes/wasm-notavail.rst
@@ -127,21 +127,23 @@ The second subclass allows for connections created by a child process:
 The following utility functions are defined:
 
 
-.. function:: Internaldate2tuple(datestr)
+.. function:: Internaldate2tuple(resp)
 
-   Parse an IMAP4 ``INTERNALDATE`` string and return corresponding local
-   time.  The return value is a :class:`time.struct_time` tuple or
-   ``None`` if the string has wrong format.
+   Parse a :term:`bytes-like object` containing an IMAP4 ``INTERNALDATE``
+   response and return the corresponding local time.  The return value is a
+   :class:`time.struct_time` tuple or ``None`` if the input has wrong format.
 
 .. function:: Int2AP(num)
 
    Converts an integer into a bytes representation using characters from the set
    [``A`` .. ``P``].
 
 
-.. function:: ParseFlags(flagstr)
+.. function:: ParseFlags(resp)
 
-   Converts an IMAP4 ``FLAGS`` response to a tuple of individual flags.
+   Converts a :term:`bytes-like object` containing an IMAP4 ``FLAGS`` response
+   to a tuple of individual flags as :class:`bytes`.  The return value is an
+   empty tuple if the input has wrong format.
 
 
 .. function:: Time2Internaldate(date_time)
@@ -450,6 +452,15 @@ An :class:`IMAP4` instance has the following methods:
    Returned data are tuples of message part envelope and data.
 
 
+.. method:: IMAP4.move(message_set, new_mailbox)
+
+   Move *message_set* messages onto end of *new_mailbox*.
+
+   The server must support the ``MOVE`` capability (:rfc:`6851`).
+
+   .. versionadded:: next
+
+
 .. method:: IMAP4.myrights(mailbox)
 
    Show my ACLs for a mailbox (i.e. the rights that I have on mailbox).
@@ -630,7 +641,7 @@ An :class:`IMAP4` instance has the following methods:
 .. method:: IMAP4.store(message_set, command, flag_list)
 
    Alters flag dispositions for messages in mailbox.  *command* is specified by
-   section 6.4.6 of :rfc:`2060` as being one of "FLAGS", "+FLAGS", or "-FLAGS",
+   section 6.4.6 of :rfc:`3501` as being one of "FLAGS", "+FLAGS", or "-FLAGS",
    optionally with a suffix of ".SILENT".
 
    For example, to set the delete flag on all messages::
@@ -644,11 +655,11 @@ An :class:`IMAP4` instance has the following methods:
 
       Creating flags containing ']' (for example: "[test]") violates
       :rfc:`3501` (the IMAP protocol).  However, imaplib has historically
-      allowed creation of such tags, and popular IMAP servers, such as Gmail,
+      allowed creation of such flags, and popular IMAP servers, such as Gmail,
       accept and produce such flags.  There are non-Python programs which also
-      create such tags.  Although it is an RFC violation and IMAP clients and
+      create such flags.  Although it is an RFC violation and IMAP clients and
       servers are supposed to be strict, imaplib still continues to allow
-      such tags to be created for backward compatibility reasons, and as of
+      such flags to be created for backward compatibility reasons, and as of
       Python 3.6, handles them if they are sent from the server, since this
       improves real-world compatibility.
 

Doc/library/urllib.robotparser.rst

@@ -24,11 +24,18 @@ structure of :file:`robots.txt` files, see :rfc:`9309`.
 .. class:: RobotFileParser(url='')
 
    This class provides methods to read, parse and answer questions about the
-   :file:`robots.txt` file at *url*.
+   :file:`robots.txt` file at *url* or a :class:`urllib.request.Request` object.
+
+   .. versionchanged:: next
+     *url* parameter can be a :class:`urllib.request.Request` object.
 
    .. method:: set_url(url)
 
-      Sets the URL referring to a :file:`robots.txt` file.
+      Sets the URL referring to a :file:`robots.txt` file or a
+      :class:`urllib.request.Request` object.
+
+      .. versionchanged:: next
+        *url* parameter can be a :class:`urllib.request.Request` object.
 
    .. method:: read()
 
@@ -102,3 +109,17 @@ class::
    True
    >>> rp.can_fetch("*", "http://www.pythontest.net/no-robots-here/")
    False
+
+
+The following example demonstrates use of a :class:`urllib.request.Request`
+object with additional user-agent headers populated::
+
+   >>> import urllib.robotparser
+   >>> import urllib.request
+   >>> rp = urllib.robotparser.RobotFileParser()
+   >>> rp.set_url(urllib.request.Request("http://www.pythontest.net/robots.txt", headers={"User-Agent": "IsraBot"}))
+   >>> rp.read()
+   >>> rp.can_fetch("*", "http://www.pythontest.net/")
+   True
+   >>> rp.can_fetch("*", "http://www.pythontest.net/no-robots-here/")
+   False

Doc/whatsnew/3.16.rst

@@ -228,6 +228,14 @@ io
   (Contributed by Marcel Martin in :gh:`90533`.)
 
 
+imaplib
+-------
+
+* Add the :meth:`~imaplib.IMAP4.move` method,
+  a wrapper for the ``MOVE`` command (:rfc:`6851`).
+  (Contributed by Serhiy Storchaka in :gh:`77508`.)
+
+
 logging
 -------
 
@@ -383,6 +391,14 @@ tkinter
   ttk version, and accepts mappings of button options as *buttons* entries.
   (Contributed by Serhiy Storchaka in :gh:`59396`.)
 
+* The :class:`!tkinter.filedialog.FileDialog` dialog and its
+  :class:`!tkinter.filedialog.LoadFileDialog` and
+  :class:`!tkinter.filedialog.SaveFileDialog` subclasses are now built from the
+  themed :mod:`tkinter.ttk` widgets by default instead of the classic
+  :mod:`tkinter` widgets, and gained a *use_ttk* parameter to select between
+  them.
+  (Contributed by Serhiy Storchaka in :gh:`59396`.)
+
 xml
 ---
 

Lib/idlelib/run.py

@@ -31,11 +31,17 @@
 import tkinter  # Use tcl and, if startup fails, messagebox.
 if not hasattr(sys.modules['idlelib.run'], 'firstrun'):
     # Undo modifications of tkinter by idlelib imports; see bpo-25507.
+    # Which of these submodules got imported (and thus added as a tkinter
+    # attribute) depends on what idlelib pulled in, so tolerate missing
+    # ones rather than assuming a fixed set; see gh-59396.
     for mod in ('simpledialog', 'messagebox', 'font',
                 'dialog', 'filedialog', 'commondialog',
                 'ttk'):
-        delattr(tkinter, mod)
-        del sys.modules['tkinter.' + mod]
+        try:
+            delattr(tkinter, mod)
+            del sys.modules['tkinter.' + mod]
+        except (AttributeError, KeyError):
+            pass
     # Avoid AttributeError if run again; see bpo-37038.
     sys.modules['idlelib.run'].firstrun = False
 

Lib/imaplib.py

@@ -1,6 +1,6 @@
 """IMAP4 client.
 
-Based on RFC 2060.
+Based on RFC 3501.
 
 Public class:           IMAP4
 Public variable:        Debug
@@ -43,8 +43,8 @@
 AllowedVersions = ('IMAP4REV1', 'IMAP4')        # Most recent first
 
 # Maximal line length when calling readline(). This is to prevent
-# reading arbitrary length lines. RFC 3501 and 2060 (IMAP 4rev1)
-# don't specify a line length. RFC 2683 suggests limiting client
+# reading arbitrary length lines. RFC 3501 (IMAP 4rev1)
+# doesn't specify a line length. RFC 2683 suggests limiting client
 # command lines to 1000 octets and that servers should be prepared
 # to accept command lines up to 8000 octets, so we used to use 10K here.
 # In the modern world (eg: gmail) the response to, for example, a
@@ -129,7 +129,9 @@
 # We compile these in _mode_xxx.
 _Literal = br'.*{(?P<size>\d+)}$'
 _Untagged_status = br'\* (?P<data>\d+) (?P<type>[A-Z-]+)( (?P<data2>.*))?'
-_control_chars = re.compile(b'[\x00-\x1F\x7F]')
+# Only NUL, CR and LF are unsafe (they cannot be represented even in
+# a quoted string); other control characters are sent quoted.
+_control_chars = re.compile(b'[\x00\r\n]')
 _non_astring_char = re.compile(br'[(){ \x00-\x1f\x7f-\xff%*\\"]')
 _non_list_char = re.compile(br'[(){ \x00-\x1f\x7f-\xff\\"]')
 _quoted = re.compile(br'"(?:[^"\\]|\\.)*+"')
@@ -787,6 +789,14 @@ def lsub(self, directory='', pattern='*'):
                                         self._list_mailbox(pattern))
         return self._untagged_response(typ, dat, name)
 
+    def move(self, message_set, new_mailbox):
+        """Move 'message_set' messages onto end of 'new_mailbox'.
+
+        (typ, [data]) = <instance>.move(message_set, new_mailbox)
+        """
+        return self._simple_command('MOVE', self._sequence_set(message_set),
+                                    self._astring(new_mailbox))
+
     def myrights(self, mailbox):
         """Show my ACLs for a mailbox (i.e. the rights that I have on mailbox).
 
@@ -1029,7 +1039,7 @@ def uid(self, command, *args):
                              (command, self.state,
                               ', '.join(Commands[command])))
         name = 'UID'
-        if command == 'COPY':
+        if command in ('COPY', 'MOVE'):
             message_set, new_mailbox = args
             args = (self._sequence_set(message_set),
                     self._astring(new_mailbox))
@@ -1170,7 +1180,7 @@ def _command(self, name, *args):
             if isinstance(arg, str):
                 arg = bytes(arg, self._encoding)
             if _control_chars.search(arg):
-                raise ValueError("Control characters not allowed in commands")
+                raise ValueError("NUL, CR and LF not allowed in commands")
             data = data + b' ' + arg
 
         literal = self.literal

Lib/test/test_imaplib.py

@@ -1173,6 +1173,35 @@ def test_uid_copy(self):
         self.assertEqual(data, [None])
         self.assertEqual(server.args, ['COPY', '4827313:4828442', '"New folder"'])
 
+    def test_move(self):
+        client, server = self._setup(make_simple_handler('MOVE'))
+        client.login('user', 'pass')
+        client.select()
+        typ, data = client.move('2:4', 'MEETING')
+        self.assertEqual(typ, 'OK')
+        self.assertEqual(data, [b'MOVE completed'])
+        self.assertEqual(server.args, ['2:4', 'MEETING'])
+
+        typ, data = client.move('2:4', 'New folder')
+        self.assertEqual(typ, 'OK')
+        self.assertEqual(data, [b'MOVE completed'])
+        self.assertEqual(server.args, ['2:4', '"New folder"'])
+
+    def test_uid_move(self):
+        client, server = self._setup(make_simple_handler('UID',
+            completed='UID MOVE completed'))
+        client.login('user', 'pass')
+        client.select()
+        typ, data = client.uid('move', '4827313:4828442', 'MEETING')
+        self.assertEqual(typ, 'OK')
+        self.assertEqual(data, [None])
+        self.assertEqual(server.args, ['MOVE', '4827313:4828442', 'MEETING'])
+
+        typ, data = client.uid('move', '4827313:4828442', 'New folder')
+        self.assertEqual(typ, 'OK')
+        self.assertEqual(data, [None])
+        self.assertEqual(server.args, ['MOVE', '4827313:4828442', '"New folder"'])
+
     def test_store(self):
         client, server = self._setup(make_simple_handler('STORE', [
             r'* 2 FETCH (FLAGS (\Deleted \Seen))',
@@ -1751,10 +1780,20 @@ def test_uppercase_command_names(self):
             client.NONEXISTENT
 
     def test_control_characters(self):
-        client, _ = self._setup(SimpleIMAPHandler)
-        for c0 in support.control_characters_c0():
+        client, server = self._setup(SimpleIMAPHandler)
+        client.login('user', 'pass')
+        for c in '\0\r\n':
             with self.assertRaises(ValueError):
-                client.login(f'user{c0}', 'pass')
+                client.select(f'a{c}b')
+        # Other control characters are valid in a quoted string and can
+        # occur in mailbox names returned by the server, so the client
+        # must be able to send them back.
+        for c in support.control_characters_c0():
+            if c in '\0\r\n':
+                continue
+            typ, _ = client.select(f'a{c}b')
+            self.assertEqual(typ, 'OK')
+            self.assertEqual(server.is_selected, [f'"a{c}b"'])
 
     # property tests
 
@@ -1895,8 +1934,8 @@ def test_connect(self):
     @threading_helper.reap_threads
     def test_bracket_flags(self):
 
-        # This violates RFC 3501, which disallows ']' characters in tag names,
-        # but imaplib has allowed producing such tags forever, other programs
+        # This violates RFC 3501, which disallows ']' characters in flags,
+        # but imaplib has allowed producing such flags forever, other programs
         # also produce them (eg: OtherInbox's Organizer app as of 20140716),
         # and Gmail, for example, accepts them and produces them.  So we
         # support them.  See issue #21815.

Lib/test/test_robotparser.py

@@ -773,6 +773,37 @@ def testServiceUnavailable(self):
         self.assertFalse(parser.can_fetch("*", url + '/path/file.html'))
 
 
+class UserAgentSiteTestCase(BaseLocalNetworkTestCase, unittest.TestCase):
+
+    class RobotHandler(BaseHTTPRequestHandler):
+        def do_GET(self):
+            if self.headers.get('User-Agent').startswith('Python-urllib'):
+                self.send_error(403, "Forbidden access")
+            else:
+                self.send_response(200)
+                self.end_headers()
+                self.wfile.write(b"User-agent: *\nDisallow:")
+
+        def log_message(self, format, *args):
+            pass
+
+    def testUserAgentFilteringSite(self):
+        addr = self.server.server_address
+        url = f'http://{socket_helper.HOST}:{addr[1]}'
+        robots_url = url + "/robots.txt"
+        file_url = url + "/document"
+        parser = urllib.robotparser.RobotFileParser()
+        parser.set_url(robots_url)
+        parser.read()
+        self.assertTrue(parser.disallow_all)
+        self.assertFalse(parser.can_fetch("*", file_url))
+        parser = urllib.robotparser.RobotFileParser()
+        parser.set_url(urllib.request.Request(robots_url, headers={'User-Agent': 'cybermapper'}))
+        parser.read()
+        self.assertFalse(parser.disallow_all)
+        self.assertTrue(parser.can_fetch("*", file_url))
+
+
 @support.requires_working_socket()
 class NetworkTestCase(unittest.TestCase):