其真正的定义是:A computer program is said to learn from experiene E with repect to some class of tasks T and performance measure P, if its performance at tasks in T,as measured by P improves with experience E.
3.Supervised Learning and Unsupervised Learning(监督学习与无监督学习)
3.1 Supervised Learning(监督学习)
What's Supervised? In Supervised Learning,we are gived a data set and already know what our correct output should look like, having the idea that there is a relationship between input and output.
Supervised learning problems are categorized into "regression" and "classification" problems.
regression problems:we are trying to predict result within a continuous output,meaning that we are trying to map intput avariable to some continuous funtion.
classification problem:we are trying to pridict result in a decreat output in other words, we are trying to map input variables into discrete categories
3.2 Unsupervised Learning(无监督学习)
Unsupervised Learning allow us to approach problem with little and no idea what our result should look like. we should derive structure from the data where we don't necessarily know the effct of variable
We can derive this structure by clustering the data based on relationships among the variables in the data.
With unsupervised learning there is no feedback based on the prediction results.
the figure below shows the machine learning structure:
4.Linear Regression Problem
4.1Cost Function
what's Cost Function? it is usually used to measure accuracy of our hypothesis function
That's need average difference of all results of the hypothesis with input from x's and the actual out y's. The math block below show:
Suppose we use X(m * (n + 1)) matrix as my input variables ,Y(m * 1) matrix as my output variable,((n + 1) * 1) as parameter of h,so the math block is:
and the J(,,…) closer to zero and more fit data!
As an example:we have some data set X = ,Y = ,we can plot it in octave as below:
suppose h = + ( = 0);as = 2,, = 1,the figure as:
and when = 2, J( = 2.3333; when = 0.5, J() = 0.58333; when = 1, J() = 0, so when J() = 0,it's more fit data.
the relationship between J() and is:
If you have more data, you can see that this is a square function as shown above.when J() approach 0,it's converges to approach minimum and more fit data.So cost function is usually used to measure accuracy of our hypothesis function.
4.2Gradient Descent
So we have our hypothesis function and we have a way of measuring how well it fits into the data. Now we need to estimate the parameters in the hypothesis function. That's where gradient descent comes in.
The gradient descent algorithm is:
repeat until convergence:
:= - J (,)
4.3 Gradient Descent For Linear Regression
when we applied to the case of linear regression, a new gradient equation that is called batch gradient descent can be derived. as below shown:
Gradient Descent for Multiple Variables:
repeat until convergence:{
}
4.4 Linear Regression Feature Scaling
we can speed up gradient descent by having each of input values in roughly the same range.because will descend quickly on small ranges and slowly on large ranges, and so will oscillate inefficiently down to the optimum when the variables are very uneven.
Ideally:
−1 ≤ x(i) ≤ 1
Two techniques to help with this are feature scaling and mean normalization:
:=
is the average of all the values for feature(i) and is the range of values(max - min),or is the standard deviation.
4.5 Polynomial Regression
Our hypothesis need not be linear(a straight line)if that does fit the data well.
we can change behavior and curve of our hypothesis function by making quadratic,cubic or square root function.
One important thing to keep in mind is, if you choose your features this way then feature scaling becomes very important.
eg. if x1 has range 1 - 1000 then range of x21 becomes 1 - 1000000 and that of x31 becomes 1 - 1000000000.
4.6 Linear Regresstion Normal Equation
Gradient descent gives one way of minizing J.Let's discuss a second way of doing so,this time performing the minimization explicitly and without resoring to an iterative algorithm.the normal equation formula is given below:
The following is a comparison of gradient descent and the normal equation:
Gradient Descent
Normal equation
Need to choose alpha
No need choose alpha
Need many iteration
No need to iterate
O(k)
O(),need to calculate inverse of X
Work well when n is large
Slow if n is large
To summarize regression of the bove:
5.Classification Regression Problem
5.1 Classifican Hypothesis Representation
Sigmoid Function or Logistic Function:
(x) = g(x)
z = x
g(z) =
The following image shows us what the sigmoid function looks like:The function g(z),shown here,map any real number to the(0,1)interval
hθ(x) will give us the probability that our output is 1. For example, hθ(x)=0.7 gives us a probability of 70% that our output is 1. Our probability that our prediction is 0 is just the complement of our probability that it is 1 (e.g. if probability that it is 1 is 70%, then the probability that it is 0 is 30%).
in order to get our discreat 0 or 1 classification,we can translate the output of the hypothesis function as follows:
(x) 0.5 -> y = 1
(x) 0.5 -> y = 0
and
,when
so
The decision boundary is the line that separates the area where y = 1 and y = 0.it is created by hypothesis function.
5.3 Logistic Regression Cost Function
We cannot use the same cost function that we use for linear regression because the Logistic Function will cause the output to be wavy, causing many local optima. In other words, it will not be a convex function.
Instead, our cost function for logistic regression looks like:
If y = 1:
If y = 0:
When y = 1,we get the follow plot for J() vs :
In contrast,when y = 0, we can get as below:
When Cost() = 0, = y.in contrast, when = y, Cost((x),y) = 0.
We can compress our function's two conditional case into case:
A vectorized implementation is:
5.4 Simplified Gradient Descent
Repeat{
}
A Vectorized implementation is:
5.5 Advance Optimization
Octave provides library to calculate optimize ,so we only use it
We first need to provide a function that evaluates the following two function for a given input value :
J()
We can write a single function that both of these
1
function [jVal,gradient] = CostFunction(theta)
2
jVal = [...code to compute J(theta)...];
3
gradient = [...code to compute derivate of J(theta)...];
4
end
then we can use octave's"fminuce()" optimization algorithm along the "optimset()" function that creates an object containing the options we want to send to "fiminuc()"
Now will approach the classification of data when we have more than categories.insted of y = {0,1} we will expand our definition so that y = {0,1…n}.
Since y = {0,1,…n},we divide our problem into n+1 binary classification problems;in each one, we predict the probability that 'y' is a member of one of our classes.
We are basically choosing one class and then lumping all the others into a single second class. We do this repeatedly, applying binary logistic regression to each case, and then use the hypothesis that returned the highest value as our prediction.such as:
To make a prediction on a new x, pick the class that maximizes .
6.Solving the Problem of Over fitting
6.1 The Problem of Overfitting
The figure on the left shown an instance of underfitting- in which the data clearly structure not captured by the model-and the figure on the right is an example of overfilling.
Underfitting,or high bias,is when the form of our hypothesis function h maps poorly to the trend of the data It is usually caused by a function that is too simple or use too few features.
Overfitting, or hight variance,,is caused by hypothesis function that fit the available data but don't generalize well to predict new data.It is usually caused by a complicated function that creates a lot of unnecessary curves and angles unrelated to the data.
This terminology is applied to both linear and logistic regression.there are two option to address the issue of overfitting
1)Reduce the number of features:
Manually select which features to keep.
Use a model selection algorithm(studied later in the course)
2) Regularization
Keep all the features,but reduce the magnitude of parameters
Regularization works well when we have a lot of slightly useful features.
6.2 Cost Function
We regularize all of our theta paramter in single summation as:
The ,or lambda,is the regularization parameter. it determines how much the cost of theta are inflated.
6.3 Regularized Linear Regression
We can apply regularization to lineat regression and logistic regression
Gradient Descent
We will modify our gradient descent function to separate out θ0 from the rest of the parameters because we do not want to penalize θ0
Repeat{
}
The term performs our regularization. With some manipulation our update rule can also be represented as:
The first term in the above equation, 1−αλm will always be less than 1. Intuitively you can see it as reducing the value of θj by some amount on every update. Notice that the second term is now exactly the same as it was before.
Normal Equation
Now let's approach regularzation using the alternate method of the non-iterative normal equation.
L is a matrix with 0 at the top left and 1's down the diagonal, with 0's everywhere else. It should have dimension (n+1)×(n+1). Intuitively, this is the identity matrix (though we are not including x0), multiplied with a single real number λ.
Recall that if m < n, then XTX is non-invertible. However, when we add the term λ⋅L, then XTX + λ⋅L becomes invertible.
Markdown is created by Daring Fireball, the original guideline is here. Its syntax, however, varies between different parsers or editors. Typora is using GitHub Flavored Markdown.
Please note that HTML fragments in markdown source will be recognized but not parsed or rendered. Also, there may be small reformatting on the original markdown source code after saving.
A paragraph is simply one or more consecutive lines of text. In markdown source code, paragraphs are separated by more than one blank lines. In Typora, you only need to press Return to create a new paragraph.
Press Shift + Return to create a single line break. However, most markdown parser will ignore single line break, to make other markdown parsers recognize your line break, you can leave two whitespace at the end of the line, or insert <br/>.
Headers
Headers use 1-6 hash characters at the start of the line, corresponding to header levels 1-6. For example:
1
# This is an H1
2
3
## This is an H2
4
5
###### This is an H6
In typora, input ‘#’s followed by title content, and press Return key will create a header.
Blockquotes
Markdown uses email-style > characters for block quoting. They are presented as:
xxxxxxxxxx
7
1
> This is a blockquote with two paragraphs. This is first paragraph.
2
>
3
> This is second pragraph.Vestibulum enim wisi, viverra nec, fringilla in, laoreet vitae, risus.
4
5
6
7
> This is another blockquote with one paragraph. There is three empty line to seperate two blockquote.
In typora, just input ‘>’ followed by quote contents a block quote is generated. Typora will insert proper ‘>’ or line break for you. Block quote inside anther block quote is allowed by adding additional levels of ‘>’.
Lists
Input * list item 1 will create an un-ordered list, the * symbol can be replace with + or -.
Input 1. list item 1 will create an ordered list, their markdown source code is like:
xxxxxxxxxx
9
1
## un-ordered list
2
* Red
3
* Green
4
* Blue
5
6
## ordered list
7
1. Red
8
2. Green
9
3.Blue
Task List
Task lists are lists with items marked as either [ ] or [x] (incomplete or complete). For example:
xxxxxxxxxx
5
1
- [ ] a task list item
2
- [ ] list syntax required
3
- [ ] normal **formatting**, @mentions, #1234 refs
4
- [ ] incomplete
5
- [x] completed
You can change the complete/incomplete state by click the checkbox before the item.
(Fenced) Code Blocks
Typora only support fences in Github Flavored Markdown. Original code blocks in markdown is not supported.
Using fences is easy: Input ``` and press return. Add an optional language identifier after ``` and we'll run it through syntax highlighting:
xxxxxxxxxx
14
1
Here's an example:
2
3
```
4
function test() {
5
console.log("notice the blank line before this function?");
6
}
7
```
8
9
syntax highlighting:
10
```ruby
11
require'redcarpet'
12
markdown=Redcarpet.new("Hello World!")
13
putsmarkdown.to_html
14
```
Math Blocks
You can render LaTeX mathematical expressions using MathJax.
Input $$, then press 'Return' key will trigger an input field which accept Tex/LaTex source. Following is an example:
In markdown source file, math block is LaTeX expression wrapped by ‘$$’ mark:
Input | First Header | Second Header | and press return key will create a table with two column.
After table is created, focus on that table will pop up a toolbar for table, where you can resize, align, or delete table. You can also use context menu to copy and add/delete column/row.
Following descriptions can be skipped, as markdown source code for tables are generated by typora automatically.
In markdown source code, they look like:
xxxxxxxxxx
4
1
| First Header | Second Header |
2
| ------------- | ------------- |
3
| Content Cell | Content Cell |
4
| Content Cell | Content Cell |
You can also include inline Markdown such as links, bold, italics, or strikethrough.
Finally, by including colons : within the header row, you can define text to be left-aligned, right-aligned, or center-aligned:
xxxxxxxxxx
5
1
| Left-Aligned | Center Aligned | Right Aligned |
2
| :------------ |:---------------:| -----:|
3
| col 3 is | some wordy text | $1600 |
4
| col 2 is | centered | $12 |
5
| zebra stripes | are neat | $1 |
A colon on the left-most side indicates a left-aligned column; a colon on the right-most side indicates a right-aligned column; a colon on both sides indicates a center-aligned column.
Footnotes
xxxxxxxxxx
3
1
You can create footnotes like this[^footnote].
2
3
[^footnote]:Here is the *text* of the **footnote**.
Mouse on the ‘footnote’ superscript to see content of the footnote.
Horizontal Rules
Input *** or --- on a blank line and press return will draw a horizontal line.
YAML Front Matter
Typora support YAML Front Matter now. Input --- at the top of the article and then press Enter will introduce one. Or insert one metadata block from the menu.
Table of Contents (TOC)
Input [toc] then press Return key will create a section for “Table of Contents” extracting all headers from one’s writing, its contents will be updated automatically.
Diagrams (Sequence, Flowchart and Mermaid)
Typora supports, sequence, flowchart and mermaid, after this feature is enabled from preference panel.
Span elements will be parsed and rendered right after your typing. Moving cursor in middle of those span elements will expand those elements into markdown source. Following will explain the syntax of those span element.
Links
Markdown supports two style of links: inline and reference.
In both styles, the link text is delimited by [square brackets].
To create an inline link, use a set of regular parentheses immediately after the link text’s closing square bracket. Inside the parentheses, put the URL where you want the link to point, along with an optional title for the link, surrounded in quotes. For example:
xxxxxxxxxx
3
1
This is [an example](http://example.com/ "Title") inline link.
2
3
[This link](http://example.net/) has no title attribute.
will produce:
This is an example inline link. (<p>This is <a href="http://example.com/" title="Title">)
This link has no title attribute. (<p><a href="http://example.net/">This link</a> has no)
Internal Links
You can set the href to headers, which will create a bookmark that allow you to jump to that section after clicking. For example:
Command(on Windows: Ctrl) + Click This link will jump to header Block Elements. To see how to write that, please move cursor or click that link with ⌘ key pressed to expand the element into markdown source.
Reference Links
Reference-style links use a second set of square brackets, inside which you place a label of your choosing to identify the link:
xxxxxxxxxx
5
1
This is [an example][id] reference-style link.
2
3
Then, anywhere in the document, you define your link label like this, on a line by itself:
The implicit link name shortcut allows you to omit the name of the link, in which case the link text itself is used as the name. Just use an empty set of square brackets — e.g., to link the word “Google” to the google.com web site, you could simply write:
xxxxxxxxxx
4
1
[Google][]
2
And then define the link:
3
4
[Google]:http://google.com/
In typora click link will expand it for editing, command+click will open the hyperlink in web browser.
URLs
Typora allows you to insert urls as links, wrapped by <brackets>.
Typora will aslo auto link standard URLs. e.g: www.google.com.
Images
Image looks similar with links, but it requires an additional ! char before the start of link. Image syntax looks like this:
xxxxxxxxxx
3
1
2
3
You are able to use drag & drop to insert image from image file or we browser. And modify the markdown source code by clicking on the image. Relative path will be used if image is in same directory or sub-directory with current editing document when drag & drop.
Markdown treats asterisks (*) and underscores (_) as indicators of emphasis. Text wrapped with one * or _ will be wrapped with an HTML <em> tag. E.g:
xxxxxxxxxx
3
1
*single asterisks*
2
3
_single underscores_
output:
single asterisks
single underscores
GFM will ignores underscores in words, which is commonly used in code and names, like this:
wow_great_stuff
do_this_and_do_that_and_another_thing.
To produce a literal asterisk or underscore at a position where it would otherwise be used as an emphasis delimiter, you can backslash escape it:
xxxxxxxxxx
1
1
\*this text is surrounded by literal asterisks\*
Typora recommends to use * symbol.
Strong
double *’s or _’s will be wrapped with an HTML <strong> tag, e.g:
xxxxxxxxxx
3
1
**double asterisks**
2
3
__double underscores__
output:
double asterisks
double underscores
Typora recommends to use ** symbol.
Code
To indicate a span of code, wrap it with backtick quotes (`). Unlike a pre-formatted code block, a code span indicates code within a normal paragraph. For example:
xxxxxxxxxx
1
1
Use the `printf()` function.
will produce:
Use the printf() function.
Strikethrough
GFM adds syntax to create strikethrough text, which is missing from standard Markdown.
~~Mistaken text.~~ becomes Mistaken text.
Underline
Underline is powered by raw HTML.
<u>Underline</u> becomes Underline.
Emoji 😆
Input emoji with syntax :smile:.
User can trigger auto-complete suggestions for emoji by pressing ESC key, or trigger it automatically after enable it on preference panel. Also, input UTF8 emoji char directly from Edit -> Emoji & Symbols from menu bar is also supported.
HTML
Typora cannot render html fragments. But typora can parse and render very limited HTML fragments, as an extension of Markdown, including:
Underline: <u>underline</u>
Image: <img src="http://www.w3.org/html/logo/img/mark-word-icon.png" width="200px" /> (And width, height attribute in HTML tag, and width, height, zoom style in style attribute will be applied.)
Most of their attributes, styles, or classes will be ignored. For other tags, typora will render them as raw HTML snippets.
But those HTML will be exported on print or export.
Inline Math
To use this feature, first, please enable it in Preference Panel -> Markdown Tab. Then use $ to wrap TeX command, for example: $\lim_{x \to \infty} \exp(-x) = 0$ will be rendered as LaTeX command.
To trigger inline preview for inline math: input “$”, then press ESC key, then input TeX command, a preview tooltip will be visible like below:
Subscript
To use this feature, first, please enable it in Preference Panel -> Markdown Tab. Then use ~ to wrap subscript content, for example: H~2~O, X~long\ text~/
Superscript
To use this feature, first, please enable it in Preference Panel -> Markdown Tab. Then use ^ to wrap superscript content, for example: X^2^.
Highlight
To use this feature, first, please enable it in Preference Panel -> Markdown Tab. Then use == to wrap superscript content, for example: ==highlight==.
]]>
<!doctype html>
<html>
<head>
<meta charset="UTF-8"><meta name="viewport" content="width=device-width initial-scale=1">
<title>Markdown Refe
Andrew Ng's Machine learning first day of the second weekhttps://blaceman.github.io/2017/03/13/Andrew-Ng-s-Machine-learning-first-day-of-the-second-week/2017-03-13T07:23:49.000Z2017-05-19T01:22:02.000ZAndrew Ng's 机器学习day threehttps://blaceman.github.io/2017/03/09/Andrew-Ng-s-机器学习day-three/2017-03-09T09:16:54.000Z2017-03-09T12:47:29.000ZParamater Learning(参数学习)
机器学习定义: A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.”()
Example: playing checkers.
E = the experience of playing many games of checkers
T = the task of playing checkers.
P = the probability that the program will win the next game.
机器学习的分类: Supervised learning and Unsupervised learning.(监督学习 无监督学习)
监督学习:we are given a data set and already know what our correct output should look like, having the idea that there is a relationship between the input and the output.(我们已经有了一个数据集,并且已经知道正确的输出应该是怎样的,得到一个输出与输入的关系)
监督学习分为:Regression(回归问题)和Classification(分类问题)
回归问题:我们试图预测连续输出的结果，这意味着我们试图将输入变量映射到一些连续函数.
分类问题:我们试图预测离散输出的结果。换句话说，我们试图将输入变量映射到离散类别。
Example 1:
Given data about the size of houses on the real estate market, try to predict their price. Price as a function of size is a continuous output, so this is a regression problem.
We could turn this example into a classification problem by instead making our output about whether the house “sells for more or less than the asking price.” Here we are classifying the houses based on price into two discrete categories.
Example 2:
(a) Regression - Given a picture of a person, we have to predict their age on the basis of the given picture(给定一个人的图片，我们必须根据给定的图片预测他们的年龄)
(b) Classification - Given a patient with a tumor, we have to predict whether the tumor is malignant or benign(给定患有肿瘤的患者，我们必须预测肿瘤是恶性的还是良性的)
无监督学习: Unsupervised learning allows us to approach problems with little or no idea what our results should look like. We can derive structure from data where we don’t necessarily know the effect of the variables.(无监督学习允许我们在不知道看起来的结果是什么的前提下,可以从不知道变量的影响的数据中导出结构)
我们可以通过基于数据中的变量之间的关系对数据进行聚类(clustering)来导出此结构。
对于无监督学习，没有基于预测结果的反馈。
Example:
Clustering: Take a collection of 1,000,000 different genes, and find a way to automatically group these genes into groups that are somehow similar or related by different variables, such as lifespan, location, roles, and so on.(获取100万个不同基因的集合，并找到一种方法来自动将这些基因分组到不同变量（如寿命，位置，角色等）中相似或相关的组中)
Non-clustering: The “Cocktail Party Algorithm”, allows you to find structure in a chaotic environment. (i.e. identifying individual voices and music from a mesh of sounds at a cocktail party).(“鸡尾酒会算法”，允许你在混乱的环境中找到结构（即，在鸡尾酒会上从声音网中识别单个声音和音乐）)
]]>
<h5 id="前言-什么是机器学习-ng老师举了几个例子谷歌网页搜索脸书或苹果的照片程序认出你的朋友电子邮件中过滤垃圾邮件的反垃圾邮件有一种科学能让计算机学习却不需要好高深的程序"><a href="#前言-什么是机器学习-NG老师举了几个例子-谷歌网页搜索-脸书或苹果的照片程
Andrew Ng's 机器学习 day twohttps://blaceman.github.io/2017/03/02/Andrew-Ng-s-机器学习-day-two/2017-03-02T10:10:21.000Z2017-03-03T17:12:36.000ZModel and Cost Function(模型与代价函数)
虽然有一个运行时函数在Mach-O文件（objc_loadModules，在objc / objc-load.h中定义）中执行Objective-C模块的动态加载，Cocoa的NSBundle类为动态加载提供了一个非常方便的界面 - 定向和集成相关服务。有关NSBundle类及其使用的信息，请参阅Foundation框架参考中的NSBundle类规范。有关Mach-O文件的信息，请参见OS X ABI Mach-O文件格式参考
- (BOOL)respondsToSelector:(SEL)aSelector
{
if ( [super respondsToSelector:aSelector] )
return YES;
else {
/* Here, test whether the aSelector message can *
* be forwarded to another object and whether that *
* object can respond to it. Return YES if it can. */
}
return NO;
}
]]>
<h4 id="最近失业有点无聊折腾一下博客然后把自己所折腾的事记录下来加深印象~"><a href="#最近失业-有点无聊-折腾一下博客-然后把自己所折腾的事记录下来-加深印象" class="headerlink" title="最近失业,有点无聊,折腾一下博客,然后把自己所
Hello worldhttps://blaceman.github.io/2016/11/29/12345/2016-11-29T09:29:34.000Z2016-12-27T06:48:52.000ZWelcome to Hexo! This is your very first post. Check documentation for more info. If you get any problems when using Hexo, you can find the answer in troubleshooting or you can ask me on GitHub.
]]>
<p>Welcome to <a href="https://hexo.io/">Hexo</a>! This is your very first post. Check <a href="https://hexo.io/docs/">documentation</a> for
2018总结https://blaceman.github.io/2001/01/02/2018总结/2001-01-01T22:23:44.000Z2001-01-01T22:23:44.000Z
If you have more data, you can see that this is a square function as shown above.when J() approach 0,it's converges to approach minimum and more fit data.So cost function is usually used to measure accuracy of our hypothesis function.
The function g(z),shown here,map any real number to the(0,1)interval
