Tag Archives: youtube-dl

youtube-dl (post_tag auto created by Wordpresser)

youtube-dl – an absurd, sad situation

What follows is my briefest introduction to “copyright”, as I limitedly understand it, followed by my personal thoughts on yesterday’s RIIA initiated DMCA takedown of the project “youtube-dl” from github.com.

The full request for the takedown of “youtube-dl”, and many of its forks, is at
https://github.com/github/dmca/blob/master/2020/10/2020-10-23-RIAA.md

Intro
“Copyright”, from an economic perspective, is a set of monopolies given to creators, to incentivize “creation”. The rational for these incentives is that creation is hard, failure-prone, and copying is relatively trivial.
However, if these monopolies were excessive, for example lasting “forever”, then creators, their heirs, or to whom the rights/monopolies were sold, would constitute a permanent bottleneck between the creation and the opportunity for society, as a whole, to benefit from it, with unrestricted freedom. Thus, the monopolies are time-limited – they have an expiration date.

There are also exceptions in the law. In the USA, “fair use” is the chapter to read, to understand exceptions. For example, showing a copyrighted video to a class of students, in an educational context, will likely be valid.
In Europe, exceptions are similar – international treaties signed by most countries have “harmonized” national copyright systems -, but include explicitly enunciated use-cases, that bypass the chance of litigation and will not require a judge to interpret particular situations as “fair use” or not, namely some learning acts at public libraries.

The creator alone has the exclusive rights to decide who/what can be done with the creation; if/how it can be modified, and if/how it can be distributed. Societies, in the so-called “Knowledge Economy” we are living in, will mostly progress fueled by better knowledge, so creators are the professionals that modern societies need and “Copyright” law must keep evolving to keep the proper balance between the creators’ rewards and the societal benefits.

The “DMCA” (Digital Millennium Copyright Act) is one of the many changes that Copyright law incorporated, in the USA. However, it is an ugly one, because until year 2000 clean reverse engineering practices would probably be legit, and since then, if for bypassing certain TPMs (Technology Protection Measures) that can “compromise” the creator, they might not be.

My thoughts
Clean reverse engineering practices usually are extraordinary innovations and should not be barred. The perverse effect of making certain TPM-defeating processes illegal, even when identified cleanly, with absolutely no access to the source intellectual property, is that the knowledge of the available bypasses will rest in the hands of the very few who do manage to find them. The chance for improvement is lost and asymmetries intensify, with solutions only available to few, definitely not available to the entire community, leaving most under the false believe that the current fruition model is the single possible one. This has fueled “bug bounty” programs, thus contributing to alternative reward systems.

These are very hard topics to discuss lightly, and this post sure is light. But, right now, I find it very negative, wrong on many levels – economical and intellectual -, damaging for all in the long-run, and intensely disrespectful for the thinkers, writers and coders involved, that RIIA is attacking years of hard labored source code developed by a community of intellectuals.

The “youtube-dl” source code has probably done nothing more than to promote the exact same artists that, allegedly, are being hurt by it. This is truly unfair. Have common sense! Some of the referenced artists themselves should take a good look at the mirror and try to assess if these tools are taking food out of their tables – what they are indirectly doing, is taking the creation pleasure out of the lives of innocents, who just enjoy creating software. Have some decency. Live and love, and let live and love.

I also tweeted about this:
https://twitter.com/my_dot_com/

A solution to DL all YouTube videos, with proxy support

I wrote a very simple solution to help people generate Youtube-DL calls, to get all videos from a user, from a channel, or just from a single URL.

The solution is available at http://arturmarques.com/forms.pub/ytdl_cmd_gen/

It is nothing more than a learning device. It is built using plain HTML and plain JavaScript, with no frameworks or extensions necessary at all.
The code is written in the simplest possible fashion, using a pedagogical approach, with long and meaningful identifiers.

The operation is very simple: the user inputs a Youtube user name, or a Youtube channel name, or a Youtube video URL, and the page is expected to output a command-line command that enables any computer capable of running https://github.com/ytdl-org/youtube-dl to download the corresponding video(s), with the best available quality.

Some neat features are the support to proxies and a memory of past downloads. If you want to download via a proxy, even with authentication, the page should handle that.

The full source code is available from this post. This is just for learning. If something breaks and the solution stops working, you should be able to fix it yourself.

<!DOCTYPE html>
<!--
File: "ytdl_cmd_gen_pub.html"
Artur Marques, https://arturmarques.com/, 2019
 -->
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Youtube-DL Command Generator Helper</title>
    <script src="./am_minimal_pattern_v2019.js"></script>
    <script src="./YTDL_CMD_GEN_PUB_SECRET_DEFAULTS.js"></script>
    <script src="./ytdl_cmd_gen_pub.js"></script>
</head>
<body>
    <h1><a href="https://github.com/ytdl-org/youtube-dl">Youtube-DL</a> command gen to DL <mark>ALL</mark> video(s)...</h1>
    <h2>...from a channel OR from a user OR from a single URL</h2>

    <form id="idFormDownloadAllVideos">
        <fieldset>
            <legend>Input <strong>channel</strong> name <mark>OR</mark> <strong>user</strong> name <mark>OR</mark> single video URL:</legend>

            <label for="idSelectIdentifierType">This is the </label>
            <select id="idSelectIdentifierType">
                <option value="c" selected>channel</option>
                <option value="u">user</option>
                <option value="s">single video URL</option>
            </select>

            <label for="idTextChannelOrUserNameOrVideoUrl"> identifier: </label>
            <input
                    value="UCUa0DzKskGo0iRYP8QzWsvA"
                    size="60"
                    type="text"
                    id="idTextChannelOrUserNameOrVideoUrl"
                    placeholder="channel OR user identifier OR single video URL">
            <br>

            <label for="idCheckUseProxy">Use a proxy? </label>
            <input
                type="checkbox"
                id="idCheckUseProxy"
                checked>

            <details open id="idDetailsProxy">
                <summary>Proxy details</summary>
                    <fieldset>
                        <legend>Proxy URL address, port and (optionally) user/login and password</legend>
                        <label for="idTextUrlProxyAddress">Proxy URL address: </label>
                        <input
                                size="60"
                                type="text"
                                id="idTextUrlProxyAddress"
                                placeholder="e.g. someregion.somevpn.com">
                        <br>

                        <label for="idNumberProxyPort">Proxy port: </label>
                        <input
                                type="number"
                                id="idNumberProxyPort"
                                placeholder="proxy port here (e.g. 80)"
                                value="80">
                        <br>

                        <label for="idCheckLoginRequired">Login required? </label>
                        <input
                                type="checkbox"
                                id="idCheckLoginRequired"
                                checked>
                        <br>

                        <details open id="idDetailsProxyUserAndPass">
                            <summary>Proxy login and password</summary>
                                <fieldset>
                                    <legend>User/login name and password</legend>
                                    <label for="idTextProxyUser">User/login name: </label>
                                    <input
                                        type="text"
                                        id="idTextProxyUser"
                                        placeholder="username">
                                    <br>

                                    <label for="idPasswordProxyPass">Password: </label>
                                    <input
                                            type="password"
                                            id="idPasswordProxyPass"
                                            placeholder="">
                                    <br>
                                </fieldset>
                        </details>
                    </fieldset>
            </details>
            <br>

            <label for="idCheckKeepMemoryOfPreviousDls">Keep memory of previous downloads (<code>--download-archive</code> option)? </label>
            <input
                    type="checkbox"
                    id="idCheckKeepMemoryOfPreviousDls"
                    checked>
            <br>

            <label for="idTextYoutubedlOptions">Youtube-dl options (do NOT use <code>--download-archive</code> if set above): </label><br>
            <input
                size="80"
                type="text"
                id="idTextYoutubedlOptions"
                placeholder="youtube-dl options"
            >
            <br>

            <input
                id="idSubmitFormDownloadAllVideos"
                type="submit"
                value="get the command to download">
        </fieldset>
    </form>
    <hr>

<h2><label for="idTaCommand">Your <mark>command<sup>*</sup></mark> will appear here:</label></h2>
    <br>
    <textarea
        cols="80"
        rows="4"
        id="idTaCommand"></textarea>
    <p><sup>*</sup>The command is intended to be called from the command-line,
        from the directory where you installed <a href="https://github.com/ytdl-org/youtube-dl">Youtube-DL</a>
        or from any other, provided your system's PATH knows where Youtube-DL is installed.<br>
        You can resize the textarea from its lower-right corner.
    </p>
</body>
</html>

/*
File: ytdl_cmd_gen_pub.js
Artur Marques, https://arturmarques.com/, 2019
 */
window.onload = boot;

const PANIC = "One or more relevant object(s) are null. Will NOT proceed.";

const YOUTUBE_CHANNEL_PREFIX = "https://www.youtube.com/channel/";
const YOUTUBE_USER_PREFIX = "https://www.youtube.com/user/";
const PROTOCOL_STRING_END_MARK = "://";
const SUB_FOLDER_FOR_CHANNELS = "c";
const SUB_FOLDER_FOR_USERS = "u";

const
    ID_FORM_DOWNLOAD_ALL_VIDEOS = "idFormDownloadAllVideos",
    ID_SELECT_IDENTIFIER_TYPE = "idSelectIdentifierType",
    ID_TEXT_CHANNEL_OR_USER_NAME_OR_SINGLE_VIDEO_URL = "idTextChannelOrUserNameOrVideoUrl",

    ID_CHECK_USER_PROXY = "idCheckUseProxy",
    ID_DETAILS_PROXY = "idDetailsProxy",
    ID_TEXT_URL_PROXY_ADDRESS = "idTextUrlProxyAddress",
    ID_NUMBER_PROXY_PORT = "idNumberProxyPort",
    ID_CHECK_LOGIN_REQUIRED = "idCheckLoginRequired",
    ID_DETAILS_PROXY_USER_AND_PASS = "idDetailsProxyUserAndPass",
    ID_TEXT_PROXY_USER = "idTextProxyUser",
    ID_PASSWORD_PROXY_PASS = "idPasswordProxyPass",

    ID_CHECK_KEEP_MEMORY_OF_PREVIOUS_DLS = "idCheckKeepMemoryOfPreviousDls",
    ID_TEXT_YOUTUBEDL_OPTIONS = "idTextYoutubedlOptions",
    ID_SUBMIT_FORM_DOWNLOAD_ALL_VIDEOS = "idSubmitFormDownloadAllVideos",
    ID_TA_COMMAND = "idTaCommand";

let oFormDownloadAllVideos, oSelectIdentifierType, oTextChannelOrUserNameOrVideoUrl,
    oCheckUseProxy, oDetailsProxy, oTextUrlProxyAddress, oNumberProxyPort, oCheckLoginRequired,
    oDetailsProxyUserAndPass, oTextProxyUser, oPasswordProxyPass,
    oCheckKeepMemoryOfPreviousDls,
    oTextYoutubedlOptions,
    oSubmitFormDownloadAllVideos,
    oTaCommand;

function boot(){
    oFormDownloadAllVideos = $(ID_FORM_DOWNLOAD_ALL_VIDEOS);
    oSelectIdentifierType = $(ID_SELECT_IDENTIFIER_TYPE);
    oTextChannelOrUserNameOrVideoUrl = $(ID_TEXT_CHANNEL_OR_USER_NAME_OR_SINGLE_VIDEO_URL);
    oCheckUseProxy = $(ID_CHECK_USER_PROXY);
    oDetailsProxy = $(ID_DETAILS_PROXY);
    oTextUrlProxyAddress = $(ID_TEXT_URL_PROXY_ADDRESS);
    oNumberProxyPort = $(ID_NUMBER_PROXY_PORT);
    oCheckLoginRequired = $(ID_CHECK_LOGIN_REQUIRED);
    oDetailsProxyUserAndPass = $(ID_DETAILS_PROXY_USER_AND_PASS);
    oTextProxyUser = $(ID_TEXT_PROXY_USER);
    oPasswordProxyPass= $(ID_PASSWORD_PROXY_PASS);
    oCheckKeepMemoryOfPreviousDls = $(ID_CHECK_KEEP_MEMORY_OF_PREVIOUS_DLS);
    oTextYoutubedlOptions = $(ID_TEXT_YOUTUBEDL_OPTIONS);
    oSubmitFormDownloadAllVideos = $(ID_SUBMIT_FORM_DOWNLOAD_ALL_VIDEOS);
    oTaCommand = $(ID_TA_COMMAND);

    let aAllRelevant = [
        oFormDownloadAllVideos, oSelectIdentifierType, oTextChannelOrUserNameOrVideoUrl,
        oCheckUseProxy, oDetailsProxy, oTextUrlProxyAddress, oNumberProxyPort, oCheckLoginRequired,
        oDetailsProxyUserAndPass, oTextProxyUser, oPasswordProxyPass,
        oCheckKeepMemoryOfPreviousDls,
        oTextYoutubedlOptions,
        oSubmitFormDownloadAllVideos,
        oTaCommand
    ];

    let bAllOK = allNotNull(aAllRelevant);

    if (bAllOK){
        setDefaultsFromSecrets();

        oTextChannelOrUserNameOrVideoUrl.oninput =
        oTextYoutubedlOptions.oninput =
        oSelectIdentifierType.onchange = actionUpdateToReflectCurrentUserInputs;

        oCheckUseProxy.onchange = function() { oDetailsProxy.open = oCheckUseProxy.checked; actionUpdateToReflectCurrentUserInputs(); };
        oCheckLoginRequired.onchange = function() { oDetailsProxyUserAndPass.open = oCheckLoginRequired.checked; actionUpdateToReflectCurrentUserInputs(); };
        oCheckKeepMemoryOfPreviousDls.onchange = function() { actionUpdateToReflectCurrentUserInputs(); };
        oNumberProxyPort.onchange = function() { actionUpdateToReflectCurrentUserInputs(); };

        oFormDownloadAllVideos.onsubmit = actionDownloadAllVideos;

        actionUpdateToReflectCurrentUserInputs();
    }//if
    else{
        window.console.log(PANIC);
        alert (PANIC);
        return;
    }//else
}//boot

function actionUpdateToReflectCurrentUserInputs (
    pTheEvent
){
    oTaCommand.value = computeYoutubedlCommandFromCurrentFormData();
}//actionUpdateToReflectCurrentUserInputs

/*
implies the availability of file "YTDL_CMD_GEN_PUB_SECRET_DEFAULTS.js"
loaded from HTML
defining the following consts:
const DEFAULT_CHECK_USE_PROXY = some boolean;
const DEFAULT_TEXT_URL_PROXY_ADDRESS = "?";
const DEFAULT_NUMBER_PROXY_PORT = some number;
const DEFAULT_CHECK_LOGIN_REQUIRED = some boolean;
const DEFAULT_TEXT_PROXY_USER = "?";
const DEFAULT_PASSWORD_PROXY_PASS = "?";
 */
function setDefaultsFromSecrets(){
    oTextYoutubedlOptions.value = DEFAULT_TEXT_YOUTUBEDL_OPTIONS;

    oCheckUseProxy.checked = DEFAULT_CHECK_USE_PROXY;
    if (oCheckUseProxy.checked){
        oDetailsProxy.open = true;
        oTextUrlProxyAddress.value = DEFAULT_TEXT_URL_PROXY_ADDRESS;
        oNumberProxyPort.value = DEFAULT_NUMBER_PROXY_PORT;

        oCheckLoginRequired.checked = DEFAULT_CHECK_LOGIN_REQUIRED;
        if (oCheckLoginRequired.checked){
            oDetailsProxyUserAndPass.open = true;
            oTextProxyUser.value = DEFAULT_TEXT_PROXY_USER;
            oPasswordProxyPass.value = DEFAULT_PASSWORD_PROXY_PASS;
        }//if
        else{
            oDetailsProxyUserAndPass.open = false;
            oTextProxyUser.value = "";
            oPasswordProxyPass.value = "";
        }
    }//if
    else{
        oDetailsProxy.open = false;
        oTextUrlProxyAddress.value = "";
        oNumberProxyPort.value = "";
    }//else
}//setDefaultsFromSecrets

function computeYoutubedlCommandFromCurrentFormData(){
    let bTypeChannel = oSelectIdentifierType.value==="c" || oSelectIdentifierType.value==="C";
    let bTypeUser = oSelectIdentifierType.value==="u" || oSelectIdentifierType.value==="U";
    let bSingleVideoUrl = oSelectIdentifierType.value==="s" || oSelectIdentifierType.value==="s";

    let strCMD ="youtube-dl ";
    if (oCheckUseProxy.checked){
        strCMD+=" --proxy ";
        if (oCheckLoginRequired.checked){
            //proxy with authentication
            let nProtocolMarkEndPos = oTextUrlProxyAddress.value.indexOf(PROTOCOL_STRING_END_MARK) + PROTOCOL_STRING_END_MARK.length;
            let strProxyUrlWithoutProtocol = oTextUrlProxyAddress.value.substr(nProtocolMarkEndPos);
            let strProtocolIncludingEndMark = oTextUrlProxyAddress.value.substr(0, nProtocolMarkEndPos);
            let strUrlWithAuthentication =
                strProtocolIncludingEndMark +
                oTextProxyUser.value+
                ":"+oPasswordProxyPass.value+"@"+
                strProxyUrlWithoutProtocol+":"+
                oNumberProxyPort.value+" ";
            strCMD += strUrlWithAuthentication;
        }//if
        else{
            //no proxy
            strCMD += oTextUrlProxyAddress.value+":"+oNumberProxyPort.value+" ";
        }//else
    }//if
    strCMD+=oTextYoutubedlOptions.value+" ";

    if (oCheckKeepMemoryOfPreviousDls.checked){
        strCMD+=" --download-archive "+"DB_"+oTextChannelOrUserNameOrVideoUrl.value+".TXT ";
    }//if

    if (bTypeUser){
        //e.g. end with ytuser:bostondynamics
        strCMD += "ytuser:"+oTextChannelOrUserNameOrVideoUrl.value;
    }//if

    if (bTypeChannel){
        //e.g. UCUa0DzKskGo0iRYP8QzWsvA => end with full channel URL
        strCMD += YOUTUBE_CHANNEL_PREFIX+oTextChannelOrUserNameOrVideoUrl.value;
    }//if

    if (bSingleVideoUrl){
        strCMD += oTextChannelOrUserNameOrVideoUrl.value;
    }
    return strCMD;
}//computeYoutubedlCommandFromCurrentFormData

function actionDownloadAllVideos(
    pTheEvent
){
    let bEmptyChannelOrUserName = oTextChannelOrUserNameOrVideoUrl.value.trim() === "";
    if (!bEmptyChannelOrUserName){
        let strFullCMD = computeYoutubedlCommandFromCurrentFormData();
        oTaCommand.innerHTML = strFullCMD;
        alert ("Done. The command is at the \"command\" text area.")
    }//if
    else{
        alert ("Empty channel or user name or single video URL! Nothing done.")
    }
    return false;
}//actionDownloadAllVideos

/*
File: am_minimal_pattern_v2019.js
Artur Marques, https://arturmarques.com/, 2019
 */
//--------------------------------------------------------------------------------
function $ (pId){return document.getElementById(pId);}

//--------------------------------------------------------------------------------
function allNotNull (pObjects){
    let bAllNotNull = true;
    //for (var o of pObjects){
    for (let idx=0; idx<pObjects.length; idx++)
    {
        let o = pObjects[idx]; //comentar esta linha se usarmos for..of
        if (o===null){
            let strMsg = "element @idx "+idx+" is null\n";
            console.log (strMsg);
        }
        bAllNotNull = bAllNotNull && (o!==null);
    }//for
    return bAllNotNull;
}//allNotNull


/*
File: YTDL_CMD_GEN_PUB_SECRET_DEFAULTS.js
*/
const DEFAULT_CHECK_USE_PROXY = true;
const DEFAULT_TEXT_URL_PROXY_ADDRESS = "https://someregion.somevpn.com"; //do NOT terminate with /
const DEFAULT_NUMBER_PROXY_PORT = 80;
const DEFAULT_CHECK_LOGIN_REQUIRED = true;
const DEFAULT_TEXT_PROXY_USER = "\"your@emailForExample.com\"";
const DEFAULT_PASSWORD_PROXY_PASS = "yourPassword";
const DEFAULT_TEXT_YOUTUBEDL_OPTIONS = " -f best -ciw --write-description --write-annotations --write-thumbnail ";