Dynamodb get item python

Dynamodb get item python DEFAULT

DynamoDB

The operation returns the attributes of one or more items from one or more tables. You identify requested items by primary key.

A single operation can retrieve up to 16 MB of data, which can contain as many as 100 items. returns a partial result if the response size limit is exceeded, the table's provisioned throughput is exceeded, or an internal processing failure occurs. If a partial result is returned, the operation returns a value for . You can use this value to retry the operation starting with the next item to get.

Warning

If you request more than 100 items, returns a with the message "Too many items requested for the BatchGetItem call."

For example, if you ask to retrieve 100 items, but each individual item is 300 KB in size, the system returns 52 items (so as not to exceed the 16 MB limit). It also returns an appropriate value so you can get the next page of results. If desired, your application can include its own logic to assemble the pages of results into one dataset.

If none of the items can be processed due to insufficient provisioned throughput on all of the tables in the request, then returns a . If at least one of the items is successfully processed, then completes successfully, while returning the keys of the unread items in .

Warning

If DynamoDB returns any unprocessed items, you should retry the batch operation on those items. However, we strongly recommend that you use an exponential backoff algorithm . If you retry the batch operation immediately, the underlying read or write requests can still fail due to throttling on the individual tables. If you delay the batch operation using exponential backoff, the individual requests in the batch are much more likely to succeed.

For more information, see Batch Operations and Error Handling in the Amazon DynamoDB Developer Guide .

By default, performs eventually consistent reads on every table in the request. If you want strongly consistent reads instead, you can set to for any or all tables.

In order to minimize response latency, retrieves items in parallel.

When designing your application, keep in mind that DynamoDB does not return items in any particular order. To help parse the response by item, include the primary key values for the items in your request in the parameter.

If a requested item does not exist, it is not returned in the result. Requests for nonexistent items consume the minimum read capacity units according to the type of read. For more information, see Working with Tables in the Amazon DynamoDB Developer Guide .

See also: AWS API Documentation

Request Syntax

response=client.batch_get_item(RequestItems={'string':{'Keys':[{'string':{'S':'string','N':'string','B':b'bytes','SS':['string',],'NS':['string',],'BS':[b'bytes',],'M':{'string':{'... recursive ...'}},'L':[{'... recursive ...'},],'NULL':True|False,'BOOL':True|False}},],'AttributesToGet':['string',],'ConsistentRead':True|False,'ProjectionExpression':'string','ExpressionAttributeNames':{'string':'string'}}},ReturnConsumedCapacity='INDEXES'|'TOTAL'|'NONE')
Parameters
  • RequestItems (dict) --

    [REQUIRED]

    A map of one or more table names and, for each table, a map that describes one or more items to retrieve from that table. Each table name can be used only once per request.

    Each element in the map of items to retrieve consists of the following:

    • - If , a strongly consistent read is used; if (the default), an eventually consistent read is used.
    • - One or more substitution tokens for attribute names in the parameter. The following are some use cases for using :
      • To access an attribute whose name conflicts with a DynamoDB reserved word.
      • To create a placeholder for repeating occurrences of an attribute name in an expression.
      • To prevent special characters in an attribute name from being misinterpreted in an expression.

    Use the # character in an expression to dereference an attribute name. For example, consider the following attribute name:

    The name of this attribute conflicts with a reserved word, so it cannot be used directly in an expression. (For the complete list of reserved words, see Reserved Words in the Amazon DynamoDB Developer Guide ). To work around this, you could specify the following for :

      You could then use this substitution in an expression, as in this example:

      Note

      Tokens that begin with the : character are expression attribute values , which are placeholders for the actual value at runtime.

      For more information about expression attribute names, see Accessing Item Attributes in the Amazon DynamoDB Developer Guide .

      • - An array of primary key attribute values that define specific items in the table. For each primary key, you must provide all of the key attributes. For example, with a simple primary key, you only need to provide the partition key value. For a composite key, you must provide both the partition key value and the sort key value.
      • - A string that identifies one or more attributes to retrieve from the table. These attributes can include scalars, sets, or elements of a JSON document. The attributes in the expression must be separated by commas. If no attribute names are specified, then all attributes are returned. If any of the requested attributes are not found, they do not appear in the result. For more information, see Accessing Item Attributes in the Amazon DynamoDB Developer Guide .
      • - This is a legacy parameter. Use instead. For more information, see AttributesToGet in the Amazon DynamoDB Developer Guide .
      • (string) --
        • (dict) --

          Represents a set of primary keys and, for each key, the attributes to retrieve from the table.

          For each primary key, you must provide all of the key attributes. For example, with a simple primary key, you only need to provide the partition key. For a composite primary key, you must provide both the partition key and the sort key.

          • Keys(list) --[REQUIRED]

            The primary key attribute values that define the items and the attributes associated with the items.

            • (dict) --
              • (string) --
                • (dict) --

                  Represents the data for an attribute.

                  Each attribute value is described as a name-value pair. The name is the data type, and the value is the data itself.

                  For more information, see Data Types in the Amazon DynamoDB Developer Guide .

                  • S(string) --

                    An attribute of type String. For example:

                  • N(string) --

                    An attribute of type Number. For example:

                    Numbers are sent across the network to DynamoDB as strings, to maximize compatibility across languages and libraries. However, DynamoDB treats them as number type attributes for mathematical operations.

                  • B(bytes) --

                    An attribute of type Binary. For example:

                  • SS(list) --

                    An attribute of type String Set. For example:

                  • NS(list) --

                    An attribute of type Number Set. For example:

                    Numbers are sent across the network to DynamoDB as strings, to maximize compatibility across languages and libraries. However, DynamoDB treats them as number type attributes for mathematical operations.

                  • BS(list) --

                    An attribute of type Binary Set. For example:

                  • M(dict) --

                    An attribute of type Map. For example:

                    • (string) --
                      • (dict) --

                        Represents the data for an attribute.

                        Each attribute value is described as a name-value pair. The name is the data type, and the value is the data itself.

                        For more information, see Data Types in the Amazon DynamoDB Developer Guide .

                  • L(list) --

                    An attribute of type List. For example:

                    • (dict) --

                      Represents the data for an attribute.

                      Each attribute value is described as a name-value pair. The name is the data type, and the value is the data itself.

                      For more information, see Data Types in the Amazon DynamoDB Developer Guide .

                  • NULL(boolean) --

                    An attribute of type Null. For example:

                  • BOOL(boolean) --

                    An attribute of type Boolean. For example:

          • AttributesToGet(list) --

            This is a legacy parameter. Use instead. For more information, see Legacy Conditional Parameters in the Amazon DynamoDB Developer Guide .

          • ConsistentRead(boolean) --

            The consistency of a read operation. If set to , then a strongly consistent read is used; otherwise, an eventually consistent read is used.

          • ProjectionExpression(string) --

            A string that identifies one or more attributes to retrieve from the table. These attributes can include scalars, sets, or elements of a JSON document. The attributes in the must be separated by commas.

            If no attribute names are specified, then all attributes will be returned. If any of the requested attributes are not found, they will not appear in the result.

            For more information, see Accessing Item Attributes in the Amazon DynamoDB Developer Guide .

          • ExpressionAttributeNames(dict) --

            One or more substitution tokens for attribute names in an expression. The following are some use cases for using :

            • To access an attribute whose name conflicts with a DynamoDB reserved word.
            • To create a placeholder for repeating occurrences of an attribute name in an expression.
            • To prevent special characters in an attribute name from being misinterpreted in an expression.

            Use the # character in an expression to dereference an attribute name. For example, consider the following attribute name:

            The name of this attribute conflicts with a reserved word, so it cannot be used directly in an expression. (For the complete list of reserved words, see Reserved Words in the Amazon DynamoDB Developer Guide ). To work around this, you could specify the following for :

              You could then use this substitution in an expression, as in this example:

              Note

              Tokens that begin with the : character are expression attribute values , which are placeholders for the actual value at runtime.

              For more information on expression attribute names, see Accessing Item Attributes in the Amazon DynamoDB Developer Guide .

      • ReturnConsumedCapacity (string) --

        Determines the level of detail about provisioned throughput consumption that is returned in the response:

        • - The response includes the aggregate for the operation, together with for each table and secondary index that was accessed. Note that some operations, such as and , do not access any indexes at all. In these cases, specifying will only return information for table(s).
        • - The response includes only the aggregate for the operation.
        • - No details are included in the response.
      Return type

      dict

      Returns

      Response Syntax

      {'Responses':{'string':[{'string':{'S':'string','N':'string','B':b'bytes','SS':['string',],'NS':['string',],'BS':[b'bytes',],'M':{'string':{'... recursive ...'}},'L':[{'... recursive ...'},],'NULL':True|False,'BOOL':True|False}},]},'UnprocessedKeys':{'string':{'Keys':[{'string':{'S':'string','N':'string','B':b'bytes','SS':['string',],'NS':['string',],'BS':[b'bytes',],'M':{'string':{'... recursive ...'}},'L':[{'... recursive ...'},],'NULL':True|False,'BOOL':True|False}},],'AttributesToGet':['string',],'ConsistentRead':True|False,'ProjectionExpression':'string','ExpressionAttributeNames':{'string':'string'}}},'ConsumedCapacity':[{'TableName':'string','CapacityUnits':123.0,'ReadCapacityUnits':123.0,'WriteCapacityUnits':123.0,'Table':{'ReadCapacityUnits':123.0,'WriteCapacityUnits':123.0,'CapacityUnits':123.0},'LocalSecondaryIndexes':{'string':{'ReadCapacityUnits':123.0,'WriteCapacityUnits':123.0,'CapacityUnits':123.0}},'GlobalSecondaryIndexes':{'string':{'ReadCapacityUnits':123.0,'WriteCapacityUnits':123.0,'CapacityUnits':123.0}}},]}

      Response Structure

      • (dict) --

        Represents the output of a operation.

        • Responses(dict) --

          A map of table name to a list of items. Each object in consists of a table name, along with a map of attribute data consisting of the data type and attribute value.

          • (string) --

            • (list) --

              • (dict) --

                • (string) --

                  • (dict) --

                    Represents the data for an attribute.

                    Each attribute value is described as a name-value pair. The name is the data type, and the value is the data itself.

                    For more information, see Data Types in the Amazon DynamoDB Developer Guide .

                    • S(string) --

                      An attribute of type String. For example:

                    • N(string) --

                      An attribute of type Number. For example:

                      Numbers are sent across the network to DynamoDB as strings, to maximize compatibility across languages and libraries. However, DynamoDB treats them as number type attributes for mathematical operations.

                    • B(bytes) --

                      An attribute of type Binary. For example:

                    • SS(list) --

                      An attribute of type String Set. For example:

                    • NS(list) --

                      An attribute of type Number Set. For example:

                      Numbers are sent across the network to DynamoDB as strings, to maximize compatibility across languages and libraries. However, DynamoDB treats them as number type attributes for mathematical operations.

                    • BS(list) --

                      An attribute of type Binary Set. For example:

                    • M(dict) --

                      An attribute of type Map. For example:

                      • (string) --

                        • (dict) --

                          Represents the data for an attribute.

                          Each attribute value is described as a name-value pair. The name is the data type, and the value is the data itself.

                          For more information, see Data Types in the Amazon DynamoDB Developer Guide .

                    • L(list) --

                      An attribute of type List. For example:

                      • (dict) --

                        Represents the data for an attribute.

                        Each attribute value is described as a name-value pair. The name is the data type, and the value is the data itself.

                        For more information, see Data Types in the Amazon DynamoDB Developer Guide .

                    • NULL(boolean) --

                      An attribute of type Null. For example:

                    • BOOL(boolean) --

                      An attribute of type Boolean. For example:

        • UnprocessedKeys(dict) --

          A map of tables and their respective keys that were not processed with the current response. The value is in the same form as , so the value can be provided directly to a subsequent operation. For more information, see in the Request Parameters section.

          Each element consists of:

          • - An array of primary key attribute values that define specific items in the table.
          • - One or more attributes to be retrieved from the table or index. By default, all attributes are returned. If a requested attribute is not found, it does not appear in the result.
          • - The consistency of a read operation. If set to , then a strongly consistent read is used; otherwise, an eventually consistent read is used.

          If there are no unprocessed keys remaining, the response contains an empty map.

          • (string) --

            • (dict) --

              Represents a set of primary keys and, for each key, the attributes to retrieve from the table.

              For each primary key, you must provide all of the key attributes. For example, with a simple primary key, you only need to provide the partition key. For a composite primary key, you must provide both the partition key and the sort key.

              • Keys(list) --

                The primary key attribute values that define the items and the attributes associated with the items.

                • (dict) --

                  • (string) --

                    • (dict) --

                      Represents the data for an attribute.

                      Each attribute value is described as a name-value pair. The name is the data type, and the value is the data itself.

                      For more information, see Data Types in the Amazon DynamoDB Developer Guide .

                      • S(string) --

                        An attribute of type String. For example:

                      • N(string) --

                        An attribute of type Number. For example:

                        Numbers are sent across the network to DynamoDB as strings, to maximize compatibility across languages and libraries. However, DynamoDB treats them as number type attributes for mathematical operations.

                      • B(bytes) --

                        An attribute of type Binary. For example:

                      • SS(list) --

                        An attribute of type String Set. For example:

                      • NS(list) --

                        An attribute of type Number Set. For example:

                        Numbers are sent across the network to DynamoDB as strings, to maximize compatibility across languages and libraries. However, DynamoDB treats them as number type attributes for mathematical operations.

                      • BS(list) --

                        An attribute of type Binary Set. For example:

                      • M(dict) --

                        An attribute of type Map. For example:

                        • (string) --

                          • (dict) --

                            Represents the data for an attribute.

                            Each attribute value is described as a name-value pair. The name is the data type, and the value is the data itself.

                            For more information, see Data Types in the Amazon DynamoDB Developer Guide .

                      • L(list) --

                        An attribute of type List. For example:

                        • (dict) --

                          Represents the data for an attribute.

                          Each attribute value is described as a name-value pair. The name is the data type, and the value is the data itself.

                          For more information, see Data Types in the Amazon DynamoDB Developer Guide .

                      • NULL(boolean) --

                        An attribute of type Null. For example:

                      • BOOL(boolean) --

                        An attribute of type Boolean. For example:

              • AttributesToGet(list) --

                This is a legacy parameter. Use instead. For more information, see Legacy Conditional Parameters in the Amazon DynamoDB Developer Guide .

              • ConsistentRead(boolean) --

                The consistency of a read operation. If set to , then a strongly consistent read is used; otherwise, an eventually consistent read is used.

              • ProjectionExpression(string) --

                A string that identifies one or more attributes to retrieve from the table. These attributes can include scalars, sets, or elements of a JSON document. The attributes in the must be separated by commas.

                If no attribute names are specified, then all attributes will be returned. If any of the requested attributes are not found, they will not appear in the result.

                For more information, see Accessing Item Attributes in the Amazon DynamoDB Developer Guide .

              • ExpressionAttributeNames(dict) --

                One or more substitution tokens for attribute names in an expression. The following are some use cases for using :

                • To access an attribute whose name conflicts with a DynamoDB reserved word.
                • To create a placeholder for repeating occurrences of an attribute name in an expression.
                • To prevent special characters in an attribute name from being misinterpreted in an expression.

                Use the # character in an expression to dereference an attribute name. For example, consider the following attribute name:

                The name of this attribute conflicts with a reserved word, so it cannot be used directly in an expression. (For the complete list of reserved words, see Reserved Words in the Amazon DynamoDB Developer Guide ). To work around this, you could specify the following for :

                  You could then use this substitution in an expression, as in this example:

                  Note

                  Tokens that begin with the : character are expression attribute values , which are placeholders for the actual value at runtime.

                  For more information on expression attribute names, see Accessing Item Attributes in the Amazon DynamoDB Developer Guide .

          • ConsumedCapacity(list) --

            The read capacity units consumed by the entire operation.

            Each element consists of:

            • - The table that consumed the provisioned throughput.
            • - The total number of capacity units consumed.
            • (dict) --

              The capacity units consumed by an operation. The data returned includes the total provisioned throughput consumed, along with statistics for the table and any indexes involved in the operation. is only returned if the request asked for it. For more information, see Provisioned Throughput in the Amazon DynamoDB Developer Guide .

              • TableName(string) --

                The name of the table that was affected by the operation.

              • CapacityUnits(float) --

                The total number of capacity units consumed by the operation.

              • ReadCapacityUnits(float) --

                The total number of read capacity units consumed by the operation.

              • WriteCapacityUnits(float) --

                The total number of write capacity units consumed by the operation.

              • Table(dict) --

                The amount of throughput consumed on the table affected by the operation.

                • ReadCapacityUnits(float) --

                  The total number of read capacity units consumed on a table or an index.

                • WriteCapacityUnits(float) --

                  The total number of write capacity units consumed on a table or an index.

                • CapacityUnits(float) --

                  The total number of capacity units consumed on a table or an index.

              • LocalSecondaryIndexes(dict) --

                The amount of throughput consumed on each local index affected by the operation.

                • (string) --

                  • (dict) --

                    Represents the amount of provisioned throughput capacity consumed on a table or an index.

                    • ReadCapacityUnits(float) --

                      The total number of read capacity units consumed on a table or an index.

                    • WriteCapacityUnits(float) --

                      The total number of write capacity units consumed on a table or an index.

                    • CapacityUnits(float) --

                      The total number of capacity units consumed on a table or an index.

              • GlobalSecondaryIndexes(dict) --

                The amount of throughput consumed on each global index affected by the operation.

                • (string) --

                  • (dict) --

                    Represents the amount of provisioned throughput capacity consumed on a table or an index.

                    • ReadCapacityUnits(float) --

                      The total number of read capacity units consumed on a table or an index.

                    • WriteCapacityUnits(float) --

                      The total number of write capacity units consumed on a table or an index.

                    • CapacityUnits(float) --

                      The total number of capacity units consumed on a table or an index.

        Exceptions

          Examples

          This example reads multiple items from the Music table using a batch of three GetItem requests. Only the AlbumTitle attribute is returned.

          response=client.batch_get_item(RequestItems={'Music':{'Keys':[{'Artist':{'S':'No One You Know',},'SongTitle':{'S':'Call Me Today',},},{'Artist':{'S':'Acme Band',},'SongTitle':{'S':'Happy Day',},},{'Artist':{'S':'No One You Know',},'SongTitle':{'S':'Scared of My Shadow',},},],'ProjectionExpression':'AlbumTitle',},},)print(response)

          Expected Output:

          {'Responses':{'Music':[{'AlbumTitle':{'S':'Somewhat Famous',},},{'AlbumTitle':{'S':'Blue Sky Blues',},},{'AlbumTitle':{'S':'Louder Than Ever',},},],},'ResponseMetadata':{'...':'...',},}
          Sours: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb.html

          Python Boto3 and Amazon DynamoDB Programming Tutorial

          DynamoDB is a speedy and flexible NoSQL database service offered by AWS (Amazon Web Service). DynamoDB is perfect for mobile apps, web apps, IoT devices, and gaming. Python has good support for DynamoDB. In this tutorial, we will use AWS Python SDK (Boto3) to perform CRUD (create, read, update, delete) operations on DynamoDB.

          Prerequisites

          Before going through this tutorial you must have prior knowledge of DynamoDB. To get started with DynamoDB, I recommend going over this article Getting Started with AWS DynamoDB.

          To get started with this tutorial, you need the following:

          • DynamoDB local: Download and configure DynamoDB. Check AWS documentation for guidelines. This version of DynamoDB is used for development purposes only. For production purposes, you should use Amazon DynamoDB Web Services.
          • Python: Download and install Python version 2.7 or later. The latest version of Python is available for download on the official website.
          • IDE: Use an IDE or a code editor of your choice. VS Code is a good option.

          Introduction to DynamoDB SDKs

          AWS provides an SDK for interacting with DynamoDB. The SDK tools are available for different programming languages. A complete list of supported programming languages is available on AWS documentation.

          In this tutorial, we will learn how to use the AWS SDK for Python (Boto3) to interact with DynamoDB. Boto3 allows Python developers to create, configure, and manage different AWS products.

          Connecting AWS Python SDK (Boto3) with DynamoDB

          Make sure you meet the prerequisites before moving forward. Install the latest version of Boto3 by running the command below. This will install the Boto3 Python dependency, which is required for our code to run.

          Now we will connect with our local instance of DynamoDB using Python. We will use the code below to do so. Note the .

          Note: For the Python code to work, we must import Boto3 dependency in our scripts using the code below.

          Boto3 can also be used to connect with online instances (production version) of AWS DynamoDB. Refer to Boto3 developer guide.

          DynamoDB Operations with Python SDK

          At this stage, we have imported the Boto3 library and have a local version of DynamoDB running. Therefore we can write Python scripts to do operations on DynamoDB. The first step will be creating a table on our DynamoDB. Before running any script, ensure that a local instance of DynamoDB is running on your computer.

          Create table

          We are going to create a table called using the method . The table has attributes as the partition key and as the sort key. Create a script and name it . Paste the code below in the script.

          In the script above, the first thing is to import the boto3 dependency. Import the dependency in every script connecting to DynamoDB. We are also connecting to DynamoDB local server. In the script, we will be defining the structure of the table.

          Only the partition key and the sort key are required. Take note of the and . defines the data types. is the maximun read and write capacity that an application can consume on a table.

          Learn more about on AWS API documentation. To run the script, enter the command below.

          Load sample data

          Let’s populate the table with some data. We will do this by loading data from a JSON file using the function . The data should be in JSON format, as shown below. Validate that the data is in a valid JSON format on JSONLint. Save the data below in a file and name it .

          Create a script named and add the code below. The code loads data from the JSON file and inserts it into the table.

          To execute the script, run the command below.

          Below is the expected response upon a successful data loading process.

          Create item

          We used the method to insert items in our table. We will create a script that inserts/creates a new item in the table . Create a script named and paste the code below.

          Run the command below to execute the script .

          We just added the item below.

          Read item

          We will read the item we just created using the method. We will need to specify the primary key of the item we want to read. In this case, the primary key of the table is a combination of a partition key and a sort key. The primary key is , and the sort key is .

          Run the command below to execute the script .

          Below is the expected output. You can confirm that the response item is the item we created previously. Using the specific primary key, we can retrieve a particular item.

          Conditions

          DynamoDB has a provision of using conditions. Conditions can be applied when updating or deleting items. We can provide a .

          If the evaluates to true, then the action is performed. Refer here for more information on condition expressions. Familiarize yourself with different DynamoDB conditions.

          Update

          Update refers to modifying a previously created item by updating the values of existing attributes, removing attributes, or adding new attributes. In this tutorial, we will update the values of existing attributes. Below is the original item and the updated item.

          Original Item

          Updated item

          We will use the method, as shown in the code below. Create a script named and add the code below.

          Run the command below to execute the script.

          Below is the expected output.

          Delete item

          To delete an item, we use the method. We can specify the primary key for the item to delete or provide a . If we use a , the item will not be deleted unless the condition is evaluated to be True.

          In this example, we will provide a primary key for the item to be deleted and provide a . The item will be deleted if the is met.

          In this example, the condition is:

          We will delete the item below:

          The item will be deleted if the value of is greater than or equal to the value provided. Create a script named and paste the code below.

          Run the command below to execute the script .

          If the is not met, the expected response will be as shown below.

          If the condition is removed or met, then the item will be deleted successfully.

          Query

          Query returns all items that match the partition key value. In this example, we will query all the data for a specific partition key. We need to specify the partition key value.

          In this case, the partition key is . We will query all the items where is equal to 10001. To learn more about DynamoDB queries, refer to the developer guide.

          Run the command below to execute the script .

          Scan

          Scan operation reads and returns all the items in the table. The method is used to scan a table. Using a , we can filter the items to be returned.

          However, the whole table will be scanned, and items not matching the will be thrown away. Create a script named and paste the code below. To learn more about DynamoDB scans, refer to the developer guide.

          The script above scans the table with no . Run the command below to execute the script scan.py. The output will be all the items in the table.

          Delete table

          To delete a table, we use the method . All we need is to specify the table name. This action is rarely performed. Create a script named and add the code below.

          Run the command below to execute the script .

          Conclusion

          We have learned how to write python scripts for interacting with AWS DynamoDB using AWS SDK for Python, Boto3. For more on Boto3 usage with DynamoDB, check AWS Boto3. Find the source code created in this tutorial on Github.


          Peer Review Contributions by: Rahul Banerjee


          About the author

          Benson Kariuki

          Benson Kariuki is a graduate computer science student. He is a passionate and solution-oriented computer scientist. His interests are Web Development with WordPress, Big Data, and Machine Learning.

          Sours: https://www.section.io/engineering-education/python-boto3-and-amazon-dynamodb-programming-tutorial/
          1. Cascade county mt gov
          2. Perfect nails glendale az
          3. Code vein 100 walkthrough

          Hands-On Examples for Working with DynamoDB, Boto3, and Python

          In this post, we’ll get hands-on with AWS DynamoDB, the Boto3 package, and Python. In my experience, I’ve found the documentation around this technology can be scattered or incomplete. I’ll do my best to explain and provide examples for some of the most common use cases.

          The easiest way to run these examples is to set up an AWS Lambda function using the Python 3.7 runtime. Also, make sure to assign a role to your function that has access to interact with the DynamoDB service.

          The Basics

          Let's start by creating a Users table with just a hash key:

          Now that we have a table, we can do an insert:

          Now, let's update our record:

          Alternatively, we could have used the same put_item method we used to create the item with. We would just need to make sure we passed in all the attributes and values in the item (not just the 1 we wanted to update).

          And finally, let's delete our record:

          Scans

          A  operation in Amazon DynamoDB reads every item in a table or a secondary index. By default, a  operation returns all of the data attributes for every item in the table or index. You can use the  parameter so that  only returns some of the attributes, rather than all of them.

          Using the same table from the above, let's go ahead and create a bunch of users.

          Basic scan example:

          We can see above that all the attributes are being returned.

          Here is an example of just scanning for all first & last names in the database:

          Scans have a 1mb limit on the data returned. If we think we’re going to exceed that, we should continue to re-scan and pass in the LastEvaluatedKey:

          Hash + Range Key

          Hash and Range Primary Key — The primary key is made of two attributes. The first attribute is the hash attribute and the second attribute is the range attribute. For example, the forum Thread table can have ForumName and Subject as its primary key, where ForumName is the hash attribute and Subject is the range attribute. DynamoDB builds an unordered hash index on the hash attribute and a sorted range index on the range attribute.

          To Demonstrate this next part, we’ll build a table for books. The title will be our hash key and author will be our range key.

          And here is an example using range key with some of the techniques we learned above:

          Global Secondary Index (GSI)

          Some applications might need to perform many kinds of queries, using a variety of different attributes as query criteria. To support these requirements, you can create one or more global secondary indexes and issue Query requests against these indexes in Amazon DynamoDB.

          To illustrate this we’re going to create an Employee table with employee_id as our hash kay and email address as our GSI.

          And here is an example of a query with an GSI:

          Local Secondary Index (LSI)

          Some applications only need to query data using the base table’s primary key. However, there might be situations where an alternative sort key would be helpful. To give your application a choice of sort keys, you can create one or more local secondary indexes on an Amazon DynamoDB table and issue Query or Scan requests against these indexes.

          To demonstrate this we’re going to create a Posts table with user_name as our hash key, title as our range key and we’ll have a LSI on user_name & subject.

          And here is an example of a query with an LSI:

          At the time of writing this get_item on LSI is not supported

          As stated in the intro, I just wanted to bring all these examples into one place. I hope that this can be a helpful resource for some of the most common use cases. 

          Sours: https://highlandsolutions.com/blog/hands-on-examples-for-working-with-dynamodb-boto3-and-python

          Step 4 - Query and Scan the Data

          You can use the method to retrieve data from a table. You must specify a partition key value. The sort key is optional.

          The primary key for the table is composed of the following:

          – The partition key. The attribute type is .

          – The sort key. The attribute type is .

          To find all movies released during a year, you need to specify only the . You can also provide the to retrieve a subset of movies based on some condition (on the sort key). For example, you can find movies released in 2014 that have a title starting with the letter “A”.

          In addition to the method, you can use the method to retrieve all the table data.

          To learn more about querying and scanning data, see Working with Queries in DynamoDB and Working with Scans in DynamoDB, respectively.


          Query - All Movies Released in a Year

          The program included in this step retrieves all movies released in the year 1985.

          1. Copy the following program and paste it into a file named .

          The Boto 3 SDK constructs a ConditionExpression for you when you use the Key and Attr functions imported from boto3.dynamodb.conditions. You can also specify a ConditionExpression as a string. For a list of available conditions for Amazon DynamoDB, see DynamoDB Conditions in AWS SDK for Python (Boto 3) Getting Started. For more information, see Condition Expressions.

          1. To run the program, enter the following command.

          The preceding program shows how to query a table by its primary key attributes. In DynamoDB, you can optionally create one or more secondary indexes on a table and query those indexes in the same way that you query a table. Secondary indexes give your applications additional flexibility by allowing queries on non-key attributes. For more information, see Improving Data Access with Secondary Indexes.


          Query - All Movies Released in a Year with Certain Titles

          The program included in this step retrieves all movies released in year 1992 with title beginning with the letter “A” through the letter “L”.

          1. Copy the following program and paste it into a file named .
          1. To run the program, enter the following command.

          Scan

          The scan method reads every item in the entire table and returns all the data in the table. You can provide an optional so that only the items matching your criteria are returned. However, the filter is applied only after the entire table has been scanned.

          The following program scans the entire table, which contains approximately 5,000 items. The scan specifies the optional filter to retrieve only the movies from the 1950s (approximately 100 items) and discard all the others.

          1. Copy the following program and paste it into a file named .

          In the code, note the following:

          • specifies the attributes you want in the scan result.

          • specifies a condition that returns only items that satisfy the condition. All other items are discarded.

          • The scan method returns a subset of the items each time, called a page. The value in the response is then passed to the scan method via the parameter. When the last page is returned, is not part of the response.

          provides name substitution. We use this because year is a reserved word in DynamoDB—you can’t use it directly in any expression, including . You can use the expression attribute name to address this. provides value substitution. You use this because you can’t use literals in any expression, including . You can use the expression attribute value to address this.

          1. To run the program, enter the following command.
          Sours: https://learn-to-code.workshop.aws/persisting_data/dynamodb/step-4.html

          Python item dynamodb get

          Getting every item from a DynamoDB table with Python

          At work, we use DynamoDB as our primary database. DynamoDB is a NoSQL database service hosted by Amazon, which we use as a persistent key-value store. Our apps make requests like “Store this document under identifier X” (PutItem) or “Give me the document stored under identifier Y” (GetItem). This sort of single document lookup is very fast in DynamoDB.

          DynamoDB is less useful if you want to do anything that involves processing documents in bulk, such as aggregating values across multiple documents, or doing a bulk update to everything in a table. There’s no built-in way to do this – you have to use the Scan operation to read everything in the table, and then write your own code to do the processing.

          I’ve written a function to get every item from a DynamoDB table many times, so I’m going to put a tidied-up version here that I can copy into future scripts and projects. If this is something you’d find useful, copy and paste it into your own code.

          Scanning in serial: simple, but slow

          The Python SDK for AWS is boto3. It includes a client for DynamoDB, and a paginator for the Scan operation that fetches results across multiple pages.

          I wrap that in a function that generates the items from the table, one at a time, as shown below. By yielding the items immediately, it avoids holding too much of the table in memory, and the calling code can start processing items immediately.

          If I want to use an extra parameter like FilterExpression, I can pass that into the function and it gets passed to the Scan.

          Scanning in parallel: faster, more complex

          The function above works fine, but it can be slow for a large table – it only reads the rows one at a time. If you want to go faster, DynamoDB has a feature called Parallel Scan. It splits the table into distinct segments, and if you run multiple workers in parallel, each reading a different segment, you can read the table much faster.

          (Long-time readers might remember I’ve previously written about using Parallel Scan in Scala.)

          This is a bit more complicated, because we have to handle the pagination logic ourselves. We also need to spin up the multiple workers, and then combine their results back together.

          The code is based on one of my recipes for concurrent.futures. It creates a future with a Scan operation for each segment of the table. When the future completes, it looks to see if there are more items to fetch in that segment – if so, it schedules another future; if not, that segment is done. It keeps doing this until it’s read the entire table.

          Depending on how much parallelism I have available, this can be many times faster than scanning in serial.

          Sours: https://alexwlchan.net/2020/05/getting-every-item-from-a-dynamodb-table-with-python/
          Complete Boto3 Tutorial to Access DynamoDB, Create, Insert, BatchWriter and More

          I recently wrote about using Node.js and the AWS SDK for JavaScript to get data from DynamoDB. In this post, I’ll take you through how to do the same thing with Python and Boto3! We’ll use both a DynamoDB client and a DynamoDB table resource in order to do many of the same read operations on the DynamoDB table. I hope this helps serve as a reference for you whenever you need to query DynamoDB with Python.

          Setting Up - Creating the Table and Loading Data

          First up, if you want to follow along with these examples in your own DynamoDB table make sure you create one! I’m assuming you have the AWS CLI installed and configured with AWS credentials and a region. But if you don’t yet, make sure to try that first. You can review the instructions from the post I mentioned above, or you can quickly create your new DynamoDB table with the AWS CLI like this:

          But, since this is a Python post, maybe you want to do this in Python instead? Well then, first make sure you have the CLI installed and configured (because we get the credentials to interact with AWS from there) and then install Boto3. Once that’s done, you can run this code right from your Python interpreter:

          Loading Data

          Now it’s time to load in the Data into the table. To do this, save the data file locally in the same directory as your other code. Then you can run this script to load the data into DynamoDB:

          This should load all the data into your new table. It might take a moment to finish but when it’s done you can start querying!

          Creating DynamoDB Client and Table Resources

          There are two main ways to use Boto3 to interact with DynamoDB. The first is called a DynamoDB Client. That’s what I used in the above code to create the DynamoDB table and to load the data in. But there is also something called a DynamoDB Table resource. This table resource can dramatically simplify some operations so it’s useful to know how the DynamoDB client and table resource differ so you can use either of them to fit your needs. In the examples below, I’ll be showing you how to use both!

          First thing, run some imports in your code to setup using both the boto3 client and table resource. You’ll notice I load in the DynamoDB conditions below. We’ll use that when we work with our table resource. Make sure you run this code before any of the examples below.

          You’ll notice that I’ve created both the DynamoDB client we’ll use in and the DynamoDB Table Resource in . Both of these can allow us to do many of the same operations to get data from the table. But we’ll use them differently in order to do so. Let’s take a look!

          #1 - Get a Single Item with the DynamoDB Client

          Below, we’re getting a Single Item from the DynamoDB table using the operation. Let’s do this with the DynamoDB client first:

          Note that with the DynamoDB client we get back the type attributes with the result. For example, we know that the is a String because the dictionary object is: . The indicates that the value inside is a string type.

          #2 - Get a Single Item with the DynamoDB Table Resource

          Now let’s see what it looks like to use the DynamoDB table resource:

          With the Table resource we get back the native Python types without that additional explanation. The dictionary only has the names of the attributes and their values: is which is a string.

          But imagine if we had a number in the data above. Say we added a ‘rating’ value that for this song was the number 10. This rating would be returned as by the DynamoDB Client and by the table resource. This means we’d have to load the Python library to interact with it. For this reason, and many other reasons related to floating point arithmetic and currency manipulation I find it easiest to store numeric values as strings without decimals and then convert them to the appropriate value later on. For example, the value is stored as a string without any decimal point in it. That means I can just do some string manipulation on the frontend to get it to a dollar value.

          #3 - Use the DynamoDB Client to Query for Items Matching a Partition Key

          Below, we’re querying the DynamoDB table for any items where the partition key is equal to the artist name of “Arturs Ardvarkian”.

          Using the DynamoDB client, we need to set a that determines what we’re looking for. In the case above, it shows us that we are querying based on the key of and that the artist attribute value is . The colon syntax is a reference that allows us to specify a variable stored in the portion of this query. It must specify the type of the attribute ( for string in this case) and the value itself .

          #4 - Use the DynamoDB Table Resource to Query for Items Matching a Partition Key

          Now let’s look at the same operation using the table resource.

          You’ll notice that the Table resource ends up having a bit more compact of a syntax using the . This lets us specify the key of the table directly and set it equal to something using . It’s also able to save us from specifying the name of the table every time we run a query which can be very useful. We do have to keep in mind that if we’re working with multiple tables we have to be more explicit in naming our table resources or else we end up wondering what table we’re interacting with when we run .

          Also, just as with the earlier single item result, we see that the table resource returns items without using the attribute type syntax that is returned by the DynamoDB client. This makes for a much more condensed response and avoids you having to reference the type/value objects of each attribute.

          #5 Querying with Partition and Sort Keys Using the DynamoDB Client

          Next, let’s see how to integrate a sort key into our query patterns. We can use the same artist as before but in this case we want to find only songs that match a sort key beginning with the letter “C”. To do this, we can use the DynamoDB operator. However, depending on which way we’re running this query we may need to set up that operator differently.

          For example, when using the DynamoDB client we would specify it inside the KeyConditionExpression string:

          In this case we use the to setup the query conditions (searching for the value and using the song to filter when it begins with a “C”). When using the DynamoDB client, we have to define the with both the type and value in order to use them in the . The and the in the are the literal name of the attributes of the table we’re interacting with. Whereas the and the are references to the we define.

          We’re also using a few operators here. First, the is used as you’d expect - to assert an equality between our partition key and the value stored in of . Then, allows us to chain an additional requirement in. We use the condition to say that we want to begin with the value stored in - .

          There are lot of other operators we could use including , , , . These work exactly as you’d expect them to with any number data types. With string values, they will sort based on the ASCII character code values.

          #6 Using (The less than operator) with the DynamoDB Client

          Here is an example of using the operator:

          When we do this, the value of the in all returned items will be less than the value contained by . In this case, because we’re working with string attributes, we’ll get song titles back like or . Both of these values start with “B”, which has an ASCII Decimal Character code of 66, whereas “C” has a character code of 67. This is why they are technically “Less than” something starting with “C”. Keep in mind that the ASCII character codes for uppercase letters like “B” and lowercase letters like “b” are different!

          #7 Using the operator with the DynamoDB Client

          We can also use the operator to specify that the returned items must have a sort key between two values. For example:

          This query will return items with song values after ‘Bz’ and before ‘D’ which because of the ASCII character code order ends up being song attribute values that start with ‘C’ like and . If we used “B” instead of “Bz” for then we would end up with song titles starting with “B” too.

          #8 Querying with Partition and Sort Keys Using the DynamoDB Table Resource

          Now let’s take a look at how we’d do some similar things with the DynamoDB table resource. Let’s look at how to do the same things we just did with the DynamoDB client. First, let’s look for songs starting with “C”. We used in the with the DynamoDB client, but with the table resource we use a slightly different .

          Rather than passing the table resource’s a string, we use two of the objects we used previously with it. Except, instead of using the method (which we could if we wanted to query for a specific item), we use . Take a look:

          This creates a compound for us that results in getting the same items as when we used in the DynamoDB client. One key difference is that with the table resource, we get the results back without attribute values.

          Overall, this object syntax can be a bit less verbose and save you time by converting data to native Python types for you automatically. However, one thing that can be somewhat confusing is wondering what the bitwise and operator is doing in the above. Well, when you take the result of ing two s you get a object that is actually passed to the and evaluated by DynamoDB. In this context, it is probably just easier to think of it as “and this other condition must also be true” rather than “let’s take the bitwise result of the two objects”.

          #9 Using - The less than method of with the DynamoDB Table Resource

          When using the table resource, we’ll use named operators instead of one of these: , , , .

          These operators are:

          • which is equivalent to - (“Greater Than”)
          • which is equivalent to - (“Greater Than or Equal To”)
          • which is equivalent to - (“Less Than”)
          • which is equivalent to - (“Less Than or Equal To”)

          Let’s follow the same earlier example and try to get values back with a that is less than ‘C’:

          This would result in the same items as the earlier query with the DynamoDB client, again with the attributes automatically put in native Python types.

          #10 Using the Method of with the DynamoDB Table Resource

          We can also still use and expect the same sort of response with native Python types. The difference here is that is a method of not a string to include in the . Here’s how we’d do it to get values between ‘Bz’ and ‘D’ (ones that probably start with ‘C’):

          If you’d like to review the full reference for how to use the comparators you can take a look here.

          What’s Next?

          These are just a few examples of how you might use either the DynamoDB table service resource or the DynamoDB Client and boto3 but there’s a variety of others! Are you curious how to write items to the table with them both? How about fetching batches of items from the table? What about conditionally writing items to a table? Or writing multiple items at the same time guaranteed? There’s a lot you can do with DynamoDB! If you’re curious about how you might do these things then sign up for my mailing list! I’ll keep you posted on my latest guides and tutorials and you can shoot me a reply to let me know what you’d like me to cover next!

          architectureserverlesspythonboto3awsdynamodbpython3
          Sours: https://www.fernandomc.com/posts/ten-examples-of-getting-data-from-dynamodb-with-python-and-boto3/

          Similar news:

          Step 3: Create, Read, Update, and Delete an Item with Python

          In this step, you perform read and write operations on an item in the table.

          To learn more about reading and writing data, see Working with Items and Attributes.

          Step 3.1: Create a New Item

          In this step, you add a new item to the table.

          1. Copy the following program and paste it into a file named .

            Note

            • The primary key is required. This code adds an item that has primary key (, ) and attributes. The attribute stores sample JSON that provides more information about the movie.

            • The class is used to print out numbers stored using the class. The Boto SDK uses the class to hold Amazon DynamoDB number values.

          2. To run the program, enter the following command.

          Step 3.2: Read an Item

          In the previous program, you added the following item to the table.

          You can use the method to read the item from the table. You must specify the primary key values so that you can read any item from if you know its and .

          1. Copy the following program and paste it into a file named .

          2. To run the program, enter the following command.

          Step 3.3: Update an Item

          You can use the method to modify an existing item. You can update values of existing attributes, add new attributes, or remove attributes.

          In this example, you perform the following updates:

          • Change the value of the existing attributes (, ).

          • Add a new list attribute () to the existing map.

          The following shows the existing item.

          The item is updated as follows.

          1. Copy the following program and paste it into a file named .

            Note

            This program uses to describe all updates you want to perform on the specified item.

            The parameter instructs DynamoDB to return only the updated attributes ().

          2. To run the program, enter the following command.

          Step 3.4: Increment an Atomic Counter

          DynamoDB supports atomic counters, which use the method to increment or decrement the value of an existing attribute without interfering with other write requests. (All write requests are applied in the order in which they are received.)

          The following program shows how to increment the for a movie. Each time you run the program, it increments this attribute by one.

          1. Copy the following program and paste it into a file named .

          2. To run the program, enter the following command.

          Step 3.5: Update an Item (Conditionally)

          The following program shows how to use with a condition. If the condition evaluates to true, the update succeeds; otherwise, the update is not performed.

          In this case, the item is updated only if there are more than three actors.

          1. Copy the following program and paste it into a file named .

          2. To run the program, enter the following command.

            The program should fail with the following message.

            The program fails because the movie has three actors in it, but the condition is checking for greater than three actors.

          3. Modify the program so that the looks like the following.

            The condition is now greater than or equal to 3 instead of greater than 3.

          4. Run the program again. The operation should now succeed.

          Step 3.6: Delete an Item

          You can use the method to delete one item by specifying its primary key. Optionally, you can provide a to prevent the item from being deleted if the condition is not met.

          In the following example, you try to delete a specific movie item if its rating is 5 or less.

          1. Copy the following program and paste it into a file named .

          2. To run the program, enter the following command.

            The program should fail with the following message.

            The program fails because the rating for this particular movie is greater than 5.

          3. Modify the program to remove the condition in .

          4. Run the program. Now, the delete succeeds because you removed the condition.

          Sours: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/GettingStarted.Python.03.html


          29118 29119 29120 29121 29122