Using NextJS, I am defining some routes in getStaticPaths by making an API call:
/**
* #dev Fetches the article route and exports the title and id to define the available routes
*/
const getAllArticles = async () => {
const result = await fetch("https://some_api_url");
const articles = await result.json();
return articles.results.map((article) => {
const articleTitle = `${article.title}`;
return {
params: {
title: articleName,
id: `${article.id}`,
},
};
});
};
/**
* #dev Defines the paths available to reach directly
*/
export async function getStaticPaths() {
const paths = await getAllArticles();
return {
paths,
fallback: false,
};
}
Everything works most of the time: I can access most of the articles, Router.push works with all URLs defined.
However, when the article name includes a special character such as &, Router.push keeps working, but copy/pasting the URL that worked from inside the app to another tab returns a page:
An unexpected error has occurred.
In the Network tab of the inspector, a 404 get request error (in Network) appears.
The component code is mostly made of API calls such as:
await API.put(`/set_article/${article.id}`, { object });
With API being defined by axios.
Any idea why it happens and how to make the getStaticPaths work with special characters?
When you transport values in URLs, they need to be URL-encoded. (When you transport values in HTML, they need to be HTML encoded. In JSON, they need to be JSON-encoded. And so on. Any text-based system that can transport structured data has an encoding scheme that you need to apply to data. URLs are not an exception.)
Turn your raw values in your client code
await API.put(`/set_article/${article.id}`)
into encoded ones
await API.put(`/set_article/${encodeURIComponent(article.id)}`)
It might be tempting, but don't pre-encode the values on the server-side. Do this on the client end, at the time you actually use them in a URL.
Related
I'm building an app for shopify and need to add the GDPR webhooks. My back end is handled using next.js and I'm writing a webhook handler to verify them. The docs havent been very helpful because they dont show how to do it with node. This is my verification function.
export function verifiedShopifyWebhookHandler(
next: (req, res, body) => Promise
): NextApiHandler {
return async (req, res) => {
const hmacHeader = req.headers['x-shopify-hmac-sha256'];
const rawBody = await getRawBody(req);
const digest = crypto.createHmac('sha256', process.env.SHOPIFY_API_SECRET).update(rawBody).digest('base64');
if (digest === hmacHeader) {
return next(req, res, rawBody);
}
const webhookId = req.headers['x-shopify-webhook-id'];
return res.status(401).end();
};
}
But I get this Error: error - InternalServerError: stream is not readable
I think it has to do with now Next.js parses the incoming requests before they are sent to my api. Any ideas?
I discovered the answer. Next.js was pre parsing the body in the context which made it so that I couldn't use the raw body parser to parse it. By setting this:
export const config = {
api: {
bodyParser: false
}
};
above the api function in the api file it prevented next from parsing it and causing the issue. I found the answer because people had the same issue integrating swipe and using the bodyParser.
I want to pass a simple string value or a JWT access token to the getServerSideProps function in NextJS, then I want to do some requests for the user from the server side.
My JWT access token is currently stored in memory (in a state).
How do I pass my JWT access token or any other string value to getServerSideProps?
I have read many times that you can pass cookies to getServerSideProps but I wanted to add this access token value to the request header or request param.
Is there any other way than cookies or querystring parameter to add custom values to the getServerSideProps function in nextjs that is executed on the server?
You can pass values to getServerSideProps if you set up a custom server. You can use their example to parse the url and then add values to the query, but note that their example uses URL.parse from node and that has been deprecated so you'll want to modify it for more recent versions of node.
A second way to handle this (still using a custom server) would be to take advantage of Express-style response locals. Using their first example on the above-linked Next page, it could look like this:
// Custom Server
...
app.prepare().then(() => {
createServer((req, res) => {
res.locals.myServerValue = "someValue"
handle(req, res)
}).listen(3000, (err) => {
if (err) throw err
console.log('> Ready on http://localhost:3000')
})
})
...
Now in getServerSideProps, you can access this value from the response object:
export const getServerSideProps = async ({ res }) => {
// Get the value from res.locals
const myServerValue = res.locals.myServerValue
// Do something with myServerValue
console.log(myServerValue) // prints "someValue"
// If desired, pass the value to the page if that's how you're using it
return {
props: {
myServerValue
}
}
}
I am using NextJS to translate a CSV of data into static pages. Each page is pages/[slug].jsx. I'm calling toLowerCase() on the slug value inside [slug].jsx getStaticPaths() and getStaticProps() functions. The generated pages are lowercase. e.g. /ab101, /ab102, /cc500 all resolve to the pages/[slug].jsx page.
Unfortunately, people might hand type the url and may use caps or mixed case for the slug value, currently resulting in a 404.
QUESTION: How can I make routing case insensitive with respect to the slug value?
UPDATE
When I return fallback: true from getStaticPaths(), my [slug].jsx file is hit even when there is not an exact path match. I can then check isFallback as illustrated by Anish Antony below.
Additionally, the items param that is passed to my page will be undefined when the page wasn't found. The router's pathname value is "/[slug]" and not the value of "slug". However, there is an asPath value which contains useful data, e.g. /mixedCaseSlug?param=value&foo=bar.
When the page renders, I check if it's a fallback. If it is, show a LOADING... message. Next will display that and call getStaticProps() to generate the "missing" page. You'll then re-render with the page data. In the event that getStaticProps couldn't get page data, I push a path that will lead to the built-in 404 page.
export default function Page({ item }) {
const { isFallback, push } = useRouter()
const hasPageData = item && Object.keys(item).length > 0
useEffect(() => {
if (!isFallback && !hasPageData) {
push('/page-not-found/error')
}
}, [isFallback, hasPageData])
const loadingMsg = <div>Loading...</div>
const notFoundMsg = <div>Page not found</div>
return isFallback ? loadingMsg : hasPageData ? <Item item={item} /> : notFoundMsg
}
I needed to update getStaticProps() to lowercase the slug param, as it may now be mixed case, but we want to find our page data. And I needed to allow for the case when there really is no data for the slug.
export async function getStaticProps({ params }) {
const { slug } = params
const item = data.find(o => o.Practice_Code.trim().toLowerCase() === slug.toLowerCase())
return {
props: {
item: item ? item : {}
}
}
}
This all seems very kludgy, so I'm still wondering if there is a better way.
NextJS routes are case sensitive.You can use fallback property in getStaticPaths to catch the routes which aren't in the same case as the one provided by default in getStaticPaths.
Edit: I have updated the answer based on the discussion with Dave.
We can give fallback:true or fallback:"blocking" , if we give fallback:true we we can show a custom component which will be displayed till the time page is loaded.For fallback:"blocking" new paths not returned by getStaticPaths will wait for the HTML to be generated,
When we give fallback:true or "blocking" static page will be generated when the user first access the site and the generated page will be served for further visits.
Sample code
export async function getStaticPaths() {
const idList = await fetchAllIds();
const paths = [
];
idList.forEach((id) => {paths.push(`/posts/${id}`)})
return { paths, fallback: true };
}
What we have note is our code in getStaticProps should be case insensitive to get the data irrespective of the one provided in url.
export async function getStaticProps({ params }) {
const { slug } = params;
try {
/// This fetch api should be able to fetch the data irrespective of the case of the slug
or we should convert it to the required case before passing it as a parameter to API
const data= await fetchSlugDetails(slug);
//Same logic should be performed if you are getting data filtered based on slug from an existing array.
return data? { props: { data} } : { notFound: true };
} catch (error) {
console.error(error);
return { notFound: true };
}
}
Note: You have to handle the notfound case and fallback case if you are using fallback:true. For fallback you can get the value isFallback from next/router while the page is being static generated
My Svelte components import readable stores like this:
import { classes, locations, schedule } from 'stores.ts'
In stores.ts, I want to build the URL for fetch dynamically using page.host from $app/stores.
// Note: this is not a Svelte component; it's stores.ts
import { readable } from 'svelte/store'
import { getStores } from '$app/stores'
const { page } = getStores()
let FQDN
page.subscribe(({ host }) => {
FQDN = host
})
const getArray = async (url) => {
const response: Response = await fetch(url)
if (!response.ok) throw new Error(`Bad response trying to retrieve from ${url}.`)
return await response.json()
}
const getReadableStore = (url: string) => readable([], set => {
getArray(`http://${FQDN}${url}`)
.then(set)
.catch(err => console.error('Failed API call:', err))
return () => {}
})
export const classes = getReadableStore('/api/class/public.json')
export const locations = getReadableStore('/api/location/public.json')
export const schedule = getReadableStore('/api/schedule/public.json')
The sixth line throws this error...
Error: Function called outside component initialization
at get_current_component (/Users/nates/dev/shy-svelte/node_modules/svelte/internal/index.js:652:15)
at Proxy.getContext (/Users/nates/dev/shy-svelte/node_modules/svelte/internal/index.js:685:12)
at Module.getStores (/.svelte-kit/dev/runtime/app/stores.js:17:26)
at eval (/src/stores.ts:6:38)
at instantiateModule (/Users/nates/dev/shy-svelte/node_modules/#sveltejs/kit/node_modules/vite/dist/node/chunks/dep-e9a16784.js:68197:166)
Two questions...
What is the correct way to get page values from $app/stores outside of the context of a component? Is this possible? Answer from below: No, this is not possible outside the context of a component.
If I'm accessing a SvelteKit site, let's say http://localhost:3000/something or https://example.com and a Svelte component loads a readable store from stores.ts, is there a way in stores.ts to determine whether the original page request that loaded the component (which loaded from stores.ts) was http or https? Answer from below: No, this is not possible in stores.ts - only from a component.
UPDATE: Based on the feedback, I'm going to set a value in my .env called VITE_WEB_URL=http://localhost:3000 and change it for the production system. This cuts down on the number of lines of code and may be a better practice (comments welcome)...
// revised stores.ts
import { readable } from 'svelte/store'
const { VITE_WEB_URL } = import.meta.env
const getArray = async (url) => {
const response: Response = await fetch(url)
if (!response.ok) throw new Error(`Bad response trying to retrieve from ${url}.`)
return await response.json()
}
const getReadableStore = (url: string) => readable([], set => {
getArray(`${VITE_WEB_URL}${url}`)
.then(set)
.catch(err => console.error('Failed API call:', err))
return () => {}
})
export const classes = getReadableStore('/api/class/public.json')
export const locations = getReadableStore('/api/location/public.json')
export const schedule = getReadableStore('/api/schedule/public.json')
Extract from https://kit.svelte.dev/docs#modules-$app-stores
Because of that, the stores are not free-floating objects: they must be accessed during component initialisation, like anything else that would be accessed with getContext.
Therefore, since the readable store is bound to the context of a svelte component, I suggest you subscribe either way ($ or .subscribe) inside the component of the SvelteKit website and then send the protocol value (http or https) as parameter when it updates so that stores.ts stores it in a variable.
However, it looks like SvelteKit does not provide the protocol value, so parse the client side window.location.href in the page subscription and then send it.
Referencing a svelte store can be done everywhere.
Using the $: shorthand syntax, however, only works within a component.
$: BASE = `http://${$page.host}`
SvelteKit appears to delegate this to fetch indeed
I'm moving a MERN project into React + MongoDB Stitch after seeing it allows for easy user authentication, quick deployment, etc.
However, I am having a hard time understanding where and how can I call a site scraping function. Previously, I web scraped in Express.js with cheerio like:
app.post("/api/getTitleAtURL", (req, res) => {
if (req.body.url) {
request(req.body.url, function(error, response, body) {
if (!error && response.statusCode == 200) {
const $ = cheerio.load(body);
const webpageTitle = $("title").text();
const metaDescription = $("meta[name=description]").attr("content");
const webpage = {
title: webpageTitle,
metaDescription: metaDescription
};
res.send(webpage);
} else {
res.status(400).send({ message: "THIS IS AN ERROR" });
}
});
}
});
But obviously with Stitch no Node & Express is needed. Is there a way to fetch another site's content without having to host a node.js application just serving that one function?
Thanks
Turns out you can build Functions in MongoDB Stitch that allows you to upload external dependencies.
However, there're limitation, for example, cheerio didn't work as an uploaded external dependency while request worked. A solution, therefore, would be to create a serverless function in AWS's lambda, and then connect mongoDB stitch to AWS lambda (mongoDB stitch can connect to many third party services, including many AWS lambda cloud services like lambda, s3, kinesis, etc).
AWS lambda allows you to upload any external dependencies, if mongoDB stitch allowed for any, we wouldn't need lambda, but stitch still needs many support. In my case, I had a node function with cheerio & request as external dependencies, to upload this to lambda: make an account, create new lambda function, and pack your node modules & code into a zip file to upload it. Your zip should look like this:
and your file containing the function should look like:
const cheerio = require("cheerio");
const request = require("request");
exports.rss = function(event, context, callback) {
request(event.requestURL, function(error, response, body) {
if (!error && response.statusCode == 200) {
const $ = cheerio.load(body);
const webpageTitle = $("title").text();
const metaDescription = $("meta[name=description]").attr("content");
const webpage = {
title: webpageTitle,
metaDescription: metaDescription
};
callback(null, webpage);
return webpage;
} else {
callback(null, {message: "THIS IS AN ERROR"})
return {message: "THIS IS AN ERROR"};
}
});
};
and in mongoDB, connect to a third party service, choose AWS, enter the secret keys you got from making an IAM amazon user. In rules -> actions, choose lambda as your API, and allow for all actions. Now, in your mongoDB stitch functions, you can connect to Lambda, and that function should look like this in my case:
exports = async function(requestURL) {
const lambda = context.services.get('getTitleAtURL').lambda("us-east-1");
const result = await lambda.Invoke({
FunctionName: "getTitleAtURL",
Payload: JSON.stringify({requestURL: requestURL})
});
console.log(result.Payload.text());
return EJSON.parse(result.Payload.text());
};
Note: this slowed down performances big time though, generally, it took twice extra time for the call to finish.