[Note Article Data Management Method] Trying CSV Output from Note API with Python

I stepped into the world of programming because I wanted to solve small inconveniences in my daily life with the power of technology.

There are several things I specifically want to achieve, such as building a system to automatically save and manage the newsletters I receive every morning in Notion, and analyzing how many people are reading my note articles.

The Necessity of Note Article Data Management

You cannot analyze how many people are reading your notes at all. Although there is a dashboard that shows the number of views for your notes, the amount of information is so small that you cannot understand anything. If you want access analysis measurement data, you need to subscribe to "note pro" for 80,000 yen per month (excluding tax). Also, if you want to install Google Analytics, you need an optional fee of 10,000 yen per month (excluding tax), so a total of 90,000 yen per month (excluding tax) is required. I want to do something about this problem!

Solution Using Python

The trigger for me to take on this problem was two articles on note. One was
satf's article which introduced how to acquire data using note's unofficial API. The other was karupoimou's article, which was also about data analysis utilizing note. After reading these articles, I decided I wanted to try it too.

Data Acquisition: Building API Requests

Today's goal is to utilize Python and note's (unofficial) API to generate a CSV file containing data such as the following:

• Article title

• Number of views

• Number of likes

• Publication date

• Article URL

By utilizing this data, I can analyze which articles are getting more response and what kind of content resonates with readers. It also becomes possible to track article performance over time.

As a first step, I learned how to access the API with Python and acquire the necessary data. Next, I created a script to output the obtained data into a CSV file in an appropriate format. This was quite a challenging part for a Python beginner, but thanks to ChatGPT's support, I managed to move forward.

Here is that script.

import requests
import pandas as pd
from io import StringIO

user_name = "◯◯◯"
contents_api_url = f"https://note.com/api/v2/creators/{user_name}/contents"
payload = {'kind': 'note'}
all_notes = []

note_count = 50  #APIから取得する総記事数
page_num = (note_count // 6) + 1 #１回のリクエストで取得できる記事数は6記事までらしい

for page in range(1, page_num + 1):
    payload['page'] = page
    res = requests.get(contents_api_url, params=payload).content
    json_str = res.decode('utf-8')
    df = pd.read_json(StringIO(json_str))
    df_notes = df["data"]["contents"]
    all_notes.extend(df_notes)

# 全ての記事をDataFrameに変換
df2 = pd.DataFrame()
for note in all_notes:
    df_n = pd.DataFrame(note.values(), index=note.keys()).T
    df2 = pd.concat([df2, df_n])

# 結果をエクセルファイルに保存
df2.to_csv('notes.csv')

Please try entering your own note ID where it says user_name = "◯◯◯".

Output to CSV File

And this is a part of what I was able to output.

Troubleshooting and Optimization

When studying programming, it is common to feel overwhelmed by outdated information and the technical barriers that come with it. Although the Note article I referenced was very useful, there were parts that did not work with the same steps due to changes in the API and Python versions. For a beginner, this can be a major hurdle.

I am a beginner who has just started studying Python. I still don't understand the syntax at all, and I can't write it yet. But I have ChatGPT with me! With this mysterious confidence, I feel like I can overcome any challenge! I intend to try various things from now on to polish my skills.

----------
The book I wrote has been published!
\Born from Twitter/
A slightly scientific home play book

Amazon Best Sellers Rank
Early Childhood Education Category Paid Top 100
It became a #1 bestseller!✨
Thank you very much!

You can also read it on Kindle Unlimited and in paperback!