Add tutorial doctest module and enable top level await (#919)

Add a tutorial module with examples that can be tested with `doctest`.

In order to simplify the examples they can be run with doctest allowing
top level await statements by adding a fixture to patch the builtins
that xdoctest uses to test code.

---------

Co-authored-by: Teemu R. <tpr@iki.fi>

README.md

@@ -171,7 +171,8 @@ Current state: {'total': 133.105, 'power': 108.223577, 'current': 0.54463, 'volt
 
 # Library usage
 
-If you want to use this library in your own project, a good starting point is to check [the documentation on discovering devices](https://python-kasa.readthedocs.io/en/latest/discover.html).
+If you want to use this library in your own project, a good starting point is [the tutorial in the documentation](https://python-kasa.readthedocs.io/en/latest/tutorial.html).
+
 You can find several code examples in the API documentation of each of the implementation base classes, check out the [documentation for the base class shared by all supported devices](https://python-kasa.readthedocs.io/en/latest/smartdevice.html).
 
 [The library design and module structure is described in a separate page](https://python-kasa.readthedocs.io/en/latest/design.html).

docs/source/conf.py

@@ -10,9 +10,10 @@
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.
 #
-# import os
+import os
-# import sys
+import sys
-# sys.path.insert(0, os.path.abspath('.'))
+
+sys.path.insert(0, os.path.abspath(".."))  # Will find modules in the docs parent
 
 
 # -- Project information -----------------------------------------------------

docs/source/design.rst

@@ -22,13 +22,13 @@ Use :func:`~kasa.Discover.discover` to perform udp-based broadcast discovery on
 This will return you a list of device instances based on the discovery replies.
 
 If the device's host is already known, you can use to construct a device instance with
-:meth:`~kasa.SmartDevice.connect()`.
+:meth:`~kasa.Device.connect()`.
 
-The :meth:`~kasa.SmartDevice.connect()` also enables support for connecting to new
+The :meth:`~kasa.Device.connect()` also enables support for connecting to new
 KASA SMART protocol and TAPO devices directly using the parameter :class:`~kasa.DeviceConfig`.
-Simply serialize the :attr:`~kasa.SmartDevice.config` property via :meth:`~kasa.DeviceConfig.to_dict()`
+Simply serialize the :attr:`~kasa.Device.config` property via :meth:`~kasa.DeviceConfig.to_dict()`
 and then deserialize it later with :func:`~kasa.DeviceConfig.from_dict()`
-and then pass it into :meth:`~kasa.SmartDevice.connect()`.
+and then pass it into :meth:`~kasa.Device.connect()`.
 
 
 .. _update_cycle:
@@ -36,7 +36,7 @@ and then pass it into :meth:`~kasa.SmartDevice.connect()`.
 Update Cycle
 ************
 
-When :meth:`~kasa.SmartDevice.update()` is called,
+When :meth:`~kasa.Device.update()` is called,
 the library constructs a query to send to the device based on :ref:`supported modules <modules>`.
 Internally, each module defines :meth:`~kasa.modules.Module.query()` to describe what they want query during the update.
 
@@ -45,7 +45,7 @@ All properties defined both in the device class and in the module classes follow
 
 While the properties are designed to provide a nice API to use for common use cases,
 you may sometimes want to access the raw, cached data as returned by the device.
-This can be done using the :attr:`~kasa.SmartDevice.internal_state` property.
+This can be done using the :attr:`~kasa.Device.internal_state` property.
 
 
 .. _modules:
@@ -53,15 +53,15 @@ This can be done using the :attr:`~kasa.SmartDevice.internal_state` property.
 Modules
 *******
 
-The functionality provided by all :class:`~kasa.SmartDevice` instances is (mostly) done inside separate modules.
+The functionality provided by all :class:`~kasa.Device` instances is (mostly) done inside separate modules.
 While the individual device-type specific classes provide an easy access for the most import features,
 you can also access individual modules through :attr:`kasa.SmartDevice.modules`.
-You can get the list of supported modules for a given device instance using :attr:`~kasa.SmartDevice.supported_modules`.
+You can get the list of supported modules for a given device instance using :attr:`~kasa.Device.supported_modules`.
 
 .. note::
 
     If you only need some module-specific information,
-    you can call the wanted method on the module to avoid using :meth:`~kasa.SmartDevice.update`.
+    you can call the wanted method on the module to avoid using :meth:`~kasa.Device.update`.
 
 Protocols and Transports
 ************************
@@ -112,10 +112,22 @@ The base exception for all library errors is :class:`KasaException <kasa.excepti
 - If the device fails to respond within a timeout the library raises a :class:`TimeoutError <kasa.exceptions.TimeoutError>`.
 - All other failures will raise the base :class:`KasaException <kasa.exceptions.KasaException>` class.
 
-API documentation for modules
+API documentation for modules and features
-*****************************
+******************************************
 
-.. automodule:: kasa.modules
+.. autoclass:: kasa.Module
+    :noindex:
+    :members:
+    :inherited-members:
+    :undoc-members:
+
+.. automodule:: kasa.interfaces
+    :noindex:
+    :members:
+    :inherited-members:
+    :undoc-members:
+
+.. autoclass:: kasa.Feature
     :noindex:
     :members:
     :inherited-members:

docs/source/device.rst

@@ -6,12 +6,12 @@ Common API
 .. contents:: Contents
    :local:
 
-SmartDevice class
+Device class
-*****************
+************
 
-The basic functionalities of all supported devices are accessible using the common :class:`SmartDevice` base class.
+The basic functionalities of all supported devices are accessible using the common :class:`Device` base class.
 
-The property accesses use the data obtained before by awaiting :func:`SmartDevice.update()`.
+The property accesses use the data obtained before by awaiting :func:`Device.update()`.
 The values are cached until the next update call. In practice this means that property accesses do no I/O and are dependent, while I/O producing methods need to be awaited.
 See :ref:`library_design` for more detailed information.
 
@@ -20,7 +20,7 @@ See :ref:`library_design` for more detailed information.
     This means that you need to use the same event loop for subsequent requests.
     The library gives a warning ("Detected protocol reuse between different event loop") to hint if you are accessing the device incorrectly.
 
-Methods changing the state of the device do not invalidate the cache (i.e., there is no implicit :func:`SmartDevice.update()` call made by the library).
+Methods changing the state of the device do not invalidate the cache (i.e., there is no implicit :func:`Device.update()` call made by the library).
 You can assume that the operation has succeeded if no exception is raised.
 These methods will return the device response, which can be useful for some use cases.
 
@@ -103,10 +103,10 @@ Currently there are three known types of encryption for TP-Link devices and two
 Devices with automatic firmware updates enabled may update to newer versions of the encryption without separate notice,
 so discovery can be helpful to determine the correct config.
 
-To connect directly pass a :class:`DeviceConfig` object to :meth:`SmartDevice.connect()`.
+To connect directly pass a :class:`DeviceConfig` object to :meth:`Device.connect()`.
 
 A :class:`DeviceConfig` can be constucted manually if you know the :attr:`DeviceConfig.connection_type` values for the device or
-alternatively the config can be retrieved from :attr:`SmartDevice.config` post discovery and then re-used.
+alternatively the config can be retrieved from :attr:`Device.config` post discovery and then re-used.
 
 Energy Consumption and Usage Statistics
 ***************************************
@@ -141,7 +141,7 @@ You can access this information using through the usage module (:class:`kasa.mod
 API documentation
 *****************
 
-.. autoclass:: SmartDevice
+.. autoclass:: Device
     :members:
     :undoc-members:
 

docs/source/discover.rst

@@ -13,7 +13,7 @@ Discovery works by sending broadcast UDP packets to two known TP-link discovery
 Port 9999 is used for legacy devices that do not use strong encryption and 20002 is for newer devices that use different
 levels of encryption.
 If a device uses port 20002 for discovery you will obtain some basic information from the device via discovery, but you
-will need to await :func:`SmartDevice.update() <kasa.SmartDevice.update()>` to get full device information.
+will need to await :func:`Device.update() <kasa.SmartDevice.update()>` to get full device information.
 Credentials will most likely be required for port 20002 devices although if the device has never been connected to the tplink
 cloud it may work without credentials.
 

docs/source/index.rst

@@ -7,8 +7,9 @@
 
    Home <self>
    cli
+   tutorial
    discover
-   smartdevice
+   device
    design
    contribute
    smartbulb

docs/source/smartbulb.rst

@@ -67,13 +67,13 @@ API documentation
     :members:
     :undoc-members:
 
-.. autoclass:: kasa.smartbulb.BehaviorMode
+.. autoclass:: kasa.iot.iotbulb.BehaviorMode
     :members:
 
-.. autoclass:: kasa.TurnOnBehaviors
+.. autoclass:: kasa.iot.iotbulb.TurnOnBehaviors
     :members:
 
 
-.. autoclass:: kasa.TurnOnBehavior
+.. autoclass:: kasa.iot.iotbulb.TurnOnBehavior
     :undoc-members:
     :members:

docs/source/tutorial.md

@@ -0,0 +1,8 @@
+# Tutorial
+
+```{eval-rst}
+.. automodule:: tutorial
+    :members:
+    :inherited-members:
+    :undoc-members:
+```

docs/tutorial.py

@@ -0,0 +1,103 @@
+# ruff: noqa
+"""
+The kasa library is fully async and methods that perform IO need to be run inside an async couroutine.
+
+These examples assume you are following the tutorial inside `asyncio REPL` (python -m asyncio) or the code
+is running inside an async function (`async def`).
+
+
+The main entry point for the API is :meth:`~kasa.Discover.discover` and
+:meth:`~kasa.Discover.discover_single` which return Device objects.
+
+Most newer devices require your TP-Link cloud username and password, but this can be omitted for older devices.
+
+>>> from kasa import Device, Discover, Credentials
+
+:func:`~kasa.Discover.discover` returns a list of devices on your network:
+
+>>> devices = await Discover.discover(credentials=Credentials("user@example.com", "great_password"))
+>>> for dev in devices:
+>>>     await dev.update()
+>>>     print(dev.host)
+127.0.0.1
+127.0.0.2
+
+:meth:`~kasa.Discover.discover_single` returns a single device by hostname:
+
+>>> dev = await Discover.discover_single("127.0.0.1", credentials=Credentials("user@example.com", "great_password"))
+>>> await dev.update()
+>>> dev.alias
+Living Room
+>>> dev.model
+L530
+>>> dev.rssi
+-52
+>>> dev.mac
+5C:E9:31:00:00:00
+
+You can update devices by calling different methods (e.g., ``set_``-prefixed ones).
+Note, that these do not update the internal state, but you need to call :meth:`~kasa.Device.update()` to query the device again.
+back to the device.
+
+>>> await dev.set_alias("Dining Room")
+>>> await dev.update()
+>>> dev.alias
+Dining Room
+
+Different groups of functionality are supported by modules which you can access via :attr:`~kasa.Device.modules` with a typed
+key from :class:`~kasa.Module`.
+
+Modules will only be available on the device if they are supported but some individual features of a module may not be available for your device.
+You can check the availability using ``is_``-prefixed properties like `is_color`.
+
+>>> from kasa import Module
+>>> Module.Light in dev.modules
+True
+>>> light = dev.modules[Module.Light]
+>>> light.brightness
+100
+>>> await light.set_brightness(50)
+>>> await dev.update()
+>>> light.brightness
+50
+>>> light.is_color
+True
+>>> if light.is_color:
+>>>     print(light.hsv)
+HSV(hue=0, saturation=100, value=50)
+
+You can test if a module is supported by using `get` to access it.
+
+>>> if effect := dev.modules.get(Module.LightEffect):
+>>>     print(effect.effect)
+>>>     print(effect.effect_list)
+>>> if effect := dev.modules.get(Module.LightEffect):
+>>>     await effect.set_effect("Party")
+>>>     await dev.update()
+>>>     print(effect.effect)
+Off
+['Off', 'Party', 'Relax']
+Party
+
+Individual pieces of functionality are also exposed via features which you can access via :attr:`~kasa.Device.features` and will only be present if they are supported.
+
+Features are similar to modules in that they provide functionality that may or may not be present.
+
+Whereas modules group functionality into a common interface, features expose a single function that may or may not be part of a module.
+
+The advantage of features is that they have a simple common interface of `id`, `name`, `value` and `set_value` so no need to learn the module API.
+
+They are useful if you want write code that dynamically adapts as new features are added to the API.
+
+>>> if auto_update := dev.features.get("auto_update_enabled"):
+>>>     print(auto_update.value)
+False
+>>> if auto_update:
+>>>     await auto_update.set_value(True)
+>>>     await dev.update()
+>>>     print(auto_update.value)
+True
+>>> for feat in dev.features.values():
+>>>     print(f"{feat.name}: {feat.value}")
+Device ID: 0000000000000000000000000000000000000000\nState: True\nSignal Level: 2\nRSSI: -52\nSSID: #MASKED_SSID#\nOverheated: False\nBrightness: 50\nCloud connection: True\nHSV: HSV(hue=0, saturation=100, value=50)\nColor temperature: 2700\nAuto update enabled: True\nUpdate available: False\nCurrent firmware version: 1.1.6 Build 240130 Rel.173828\nAvailable firmware version: 1.1.6 Build 240130 Rel.173828\nLight effect: Party\nSmooth transition on: 2\nSmooth transition off: 2\nTime: 2024-02-23 02:40:15+01:00
+"""

kasa/__init__.py

@@ -1,12 +1,12 @@
 """Python interface for TP-Link's smart home devices.
 
-All common, shared functionalities are available through `SmartDevice` class::
+All common, shared functionalities are available through `Device` class::
 
-    x = SmartDevice("192.168.1.1")
+>>> from kasa import Discover
-    print(x.sys_info)
+>>> x = await Discover.discover_single("192.168.1.1")
+>>> print(x.model)
 
-For device type specific actions `SmartBulb`, `SmartPlug`, or `SmartStrip`
+For device type specific actions `modules` and `features` should be used instead.
- should be used instead.
 
 Module-specific errors are raised as `KasaException` and are expected
 to be handled by the user of the library.

kasa/deviceconfig.py

@@ -150,7 +150,7 @@ class DeviceConfig:
     credentials: Optional[Credentials] = None
     #: Credentials hash for devices requiring authentication.
     #: If credentials are also supplied they take precendence over credentials_hash.
-    #: Credentials hash can be retrieved from :attr:`SmartDevice.credentials_hash`
+    #: Credentials hash can be retrieved from :attr:`Device.credentials_hash`
     credentials_hash: Optional[str] = None
     #: The protocol specific type of connection.  Defaults to the legacy type.
     batch_size: Optional[int] = None

kasa/discover.py

@@ -270,10 +270,10 @@ class Discover:
         you can use *target* parameter to specify the network for discovery.
 
         If given, `on_discovered` coroutine will get awaited with
-        a :class:`SmartDevice`-derived object as parameter.
+        a :class:`Device`-derived object as parameter.
 
         The results of the discovery are returned as a dict of
-        :class:`SmartDevice`-derived objects keyed with IP addresses.
+        :class:`Device`-derived objects keyed with IP addresses.
         The devices are already initialized and all but emeter-related properties
         can be accessed directly.
 
@@ -332,7 +332,7 @@ class Discover:
         """Discover a single device by the given IP address.
 
         It is generally preferred to avoid :func:`discover_single()` and
-        use :meth:`SmartDevice.connect()` instead as it should perform better when
+        use :meth:`Device.connect()` instead as it should perform better when
         the WiFi network is congested or the device is not responding
         to discovery requests.
 

kasa/iot/iotbulb.py

@@ -180,7 +180,7 @@ class IotBulb(IotDevice):
         >>> bulb.presets
         [LightPreset(index=0, brightness=50, hue=0, saturation=0, color_temp=2700, custom=None, id=None, mode=None), LightPreset(index=1, brightness=100, hue=0, saturation=75, color_temp=0, custom=None, id=None, mode=None), LightPreset(index=2, brightness=100, hue=120, saturation=75, color_temp=0, custom=None, id=None, mode=None), LightPreset(index=3, brightness=100, hue=240, saturation=75, color_temp=0, custom=None, id=None, mode=None)]
 
-        To modify an existing preset, pass :class:`~kasa.smartbulb.LightPreset`
+        To modify an existing preset, pass :class:`~kasa.interfaces.light.LightPreset`
         instance to :func:`save_preset` method:
 
         >>> preset = bulb.presets[0]

kasa/iot/iotplug.py

@@ -41,7 +41,7 @@ class IotPlug(IotDevice):
         >>> plug.led
         True
 
-    For more examples, see the :class:`SmartDevice` class.
+    For more examples, see the :class:`Device` class.
     """
 
     def __init__(

kasa/iot/iotstrip.py

@@ -83,7 +83,7 @@ class IotStrip(IotDevice):
         >>> strip.is_on
         True
 
-    For more examples, see the :class:`SmartDevice` class.
+    For more examples, see the :class:`Device` class.
     """
 
     def __init__(

kasa/tests/fixtures/smart/L530E(EU)_3.0_1.1.6.json

@@ -175,7 +175,7 @@
         "longitude": 0,
         "mac": "5C-E9-31-00-00-00",
         "model": "L530",
-        "nickname": "I01BU0tFRF9OQU1FIw==",
+        "nickname": "TGl2aW5nIFJvb20=",
         "oem_id": "00000000000000000000000000000000",
         "overheated": false,
         "region": "Europe/Berlin",

kasa/tests/test_readme_examples.py

@@ -1,7 +1,9 @@
 import asyncio
 
+import pytest
 import xdoctest
 
+from kasa import Discover
 from kasa.tests.conftest import get_device_for_fixture_protocol
 
 
@@ -67,3 +69,61 @@ def test_discovery_examples(mocker):
     mocker.patch("kasa.discover.Discover.discover", return_value=[p])
     res = xdoctest.doctest_module("kasa.discover", "all")
     assert not res["failed"]
+
+
+def test_tutorial_examples(mocker, top_level_await):
+    """Test discovery examples."""
+    a = asyncio.run(
+        get_device_for_fixture_protocol("L530E(EU)_3.0_1.1.6.json", "SMART")
+    )
+    b = asyncio.run(get_device_for_fixture_protocol("HS110(EU)_1.0_1.2.5.json", "IOT"))
+    a.host = "127.0.0.1"
+    b.host = "127.0.0.2"
+
+    # Note autospec does not work for staticmethods in python < 3.12
+    # https://github.com/python/cpython/issues/102978
+    mocker.patch(
+        "kasa.discover.Discover.discover_single", return_value=a, autospec=True
+    )
+    mocker.patch.object(Discover, "discover", return_value=[a, b], autospec=True)
+    res = xdoctest.doctest_module("docs/tutorial.py", "all")
+    assert not res["failed"]
+
+
+@pytest.fixture
+def top_level_await(mocker):
+    """Fixture to enable top level awaits in doctests.
+
+    Uses the async exec feature of python to patch the builtins xdoctest uses.
+    See https://github.com/python/cpython/issues/78797
+    """
+    import ast
+    from inspect import CO_COROUTINE
+
+    orig_exec = exec
+    orig_eval = eval
+    orig_compile = compile
+
+    def patch_exec(source, globals=None, locals=None, /, **kwargs):
+        if source.co_flags & CO_COROUTINE == CO_COROUTINE:
+            asyncio.run(orig_eval(source, globals, locals))
+        else:
+            orig_exec(source, globals, locals, **kwargs)
+
+    def patch_eval(source, globals=None, locals=None, /, **kwargs):
+        if source.co_flags & CO_COROUTINE == CO_COROUTINE:
+            return asyncio.run(orig_eval(source, globals, locals, **kwargs))
+        else:
+            return orig_eval(source, globals, locals, **kwargs)
+
+    def patch_compile(
+        source, filename, mode, flags=0, dont_inherit=False, optimize=-1, **kwargs
+    ):
+        flags |= ast.PyCF_ALLOW_TOP_LEVEL_AWAIT
+        return orig_compile(
+            source, filename, mode, flags, dont_inherit, optimize, **kwargs
+        )
+
+    mocker.patch("builtins.eval", side_effect=patch_eval)
+    mocker.patch("builtins.exec", side_effect=patch_exec)
+    mocker.patch("builtins.compile", side_effect=patch_compile)