Skip to content

Commit e5a9508

Browse files
committed
docs modifications + remove None default of the file_sender generator
1 parent 68ce6bb commit e5a9508

File tree

2 files changed

+75
-35
lines changed

2 files changed

+75
-35
lines changed

docs/usage/file_upload.rst

Lines changed: 74 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ In order to upload a single file, you need to:
1919
2020
transport = AIOHTTPTransport(url='YOUR_URL')
2121
22-
client = Client(transport=sample_transport)
22+
client = Client(transport=transport)
2323
2424
query = gql('''
2525
mutation($file: Upload!) {
@@ -46,7 +46,7 @@ It is also possible to upload multiple files using a list.
4646
4747
transport = AIOHTTPTransport(url='YOUR_URL')
4848
49-
client = Client(transport=sample_transport)
49+
client = Client(transport=transport)
5050
5151
query = gql('''
5252
mutation($files: [Upload!]!) {
@@ -69,51 +69,45 @@ It is also possible to upload multiple files using a list.
6969
f2.close()
7070
7171
72-
Aiohttp StreamReader
73-
--------------------
72+
Streaming
73+
---------
7474

75-
In order to upload a aiohttp StreamReader, you need to:
75+
If you use the above methods to send files, then the entire contents of the files
76+
must be loaded in memory before the files are sent.
77+
If the files are not too big and you have enough RAM, it is not a problem.
78+
On another hand if you want to avoid using too much memory, then it is better
79+
to read the files and send them in small chunks so that the entire file contents
80+
don't have to be in memory at once.
7681

77-
* get response from aiohttp request and then get StreamReader from `resp.content`
78-
* provide the StreamReader to the `variable_values` argument of `execute`
79-
* set the `upload_files` argument to True
82+
We provide methods to do that for two different uses cases:
8083

84+
* Sending local files
85+
* Streaming downloaded files from an external URL to the GraphQL API
8186

82-
.. code-block:: python
87+
Streaming local files
88+
^^^^^^^^^^^^^^^^^^^^^
8389

84-
async with ClientSession() as client:
85-
async with client.get('YOUR_URL') as resp:
86-
transport = AIOHTTPTransport(url='YOUR_URL')
87-
client = Client(transport=transport)
88-
query = gql('''
89-
mutation($file: Upload!) {
90-
singleUpload(file: $file) {
91-
id
92-
}
93-
}
94-
''')
90+
aiohttp allows to upload files using an asynchronous generator.
91+
See `Streaming uploads on aiohttp docs`_.
9592

96-
params = {"file": resp.content}
9793

98-
result = client.execute(
99-
query, variable_values=params, upload_files=True
100-
)
94+
In order to stream local files, instead of providing opened files to the
95+
`variables_values` argument of `execute`, you need to provide an async generator
96+
which will provide parts of the files.
10197

102-
Asynchronous Generator
103-
----------------------
98+
You can use `aiofiles`_
99+
to read the files in chunks and create this asynchronous generator.
104100

105-
In order to upload a single file use asynchronous generator(https://docs.aiohttp.org/en/stable/client_quickstart.html#streaming-uploads), you need to:
101+
.. _Streaming uploads on aiohttp docs: https://docs.aiohttp.org/en/stable/client_quickstart.html#streaming-uploads
102+
.. _aiofiles: https://github.com/Tinche/aiofiles
106103

107-
* сreate a asynchronous generator
108-
* set the generator as a variable value in the mutation
109-
* provide the opened file to the `variable_values` argument of `execute`
110-
* set the `upload_files` argument to True
104+
Example:
111105

112106
.. code-block:: python
113107
114108
transport = AIOHTTPTransport(url='YOUR_URL')
115109
116-
client = Client(transport=sample_transport)
110+
client = Client(transport=transport)
117111
118112
query = gql('''
119113
mutation($file: Upload!) {
@@ -123,7 +117,7 @@ In order to upload a single file use asynchronous generator(https://docs.aiohttp
123117
}
124118
''')
125119
126-
async def file_sender(file_name=None):
120+
async def file_sender(file_name):
127121
async with aiofiles.open(file_name, 'rb') as f:
128122
chunk = await f.read(64*1024)
129123
while chunk:
@@ -134,4 +128,50 @@ In order to upload a single file use asynchronous generator(https://docs.aiohttp
134128
135129
result = client.execute(
136130
query, variable_values=params, upload_files=True
137-
)
131+
)
132+
133+
Streaming downloaded files
134+
^^^^^^^^^^^^^^^^^^^^^^^^^^
135+
136+
If the file you want to upload to the GraphQL API is not present locally
137+
and needs to be downloaded from elsewhere, then it is possible to chain the download
138+
and the upload in order to limit the amout of memory used.
139+
140+
Because the `content` attribute of an aiohttp response is a `StreamReader`
141+
(it provides an async iterator protocol), you can chain the download and the upload
142+
together.
143+
144+
In order to do that, you need to:
145+
146+
* get the response from an aiohttp request and then get the StreamReader instance
147+
from `resp.content`
148+
* provide the StreamReader instance to the `variable_values` argument of `execute`
149+
150+
Example:
151+
152+
.. code-block:: python
153+
154+
# First request to download your file with aiohttp
155+
async with aiohttp.ClientSession() as http_client:
156+
async with http_client.get('YOUR_DOWNLOAD_URL') as resp:
157+
158+
# We now have a StreamReader instance in resp.content
159+
# and we provide it to the variable_values argument of execute
160+
161+
transport = AIOHTTPTransport(url='YOUR_GRAPHQL_URL')
162+
163+
client = Client(transport=transport)
164+
165+
query = gql('''
166+
mutation($file: Upload!) {
167+
singleUpload(file: $file) {
168+
id
169+
}
170+
}
171+
''')
172+
173+
params = {"file": resp.content}
174+
175+
result = client.execute(
176+
query, variable_values=params, upload_files=True
177+
)

tests/test_aiohttp.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -641,7 +641,7 @@ async def test_aiohttp_async_generator_upload(event_loop, aiohttp_server):
641641

642642
file_path = test_file.filename
643643

644-
async def file_sender(file_name=None):
644+
async def file_sender(file_name):
645645
async with aiofiles.open(file_name, "rb") as f:
646646
chunk = await f.read(64 * 1024)
647647
while chunk:

0 commit comments

Comments
 (0)